From patchwork Wed Jul 3 13:56:30 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: shejialuo X-Patchwork-Id: 13722251 Received: from mail-pj1-f53.google.com (mail-pj1-f53.google.com [209.85.216.53]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 71CE685624 for ; Wed, 3 Jul 2024 13:56:32 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.53 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1720014993; cv=none; b=NUvSCXlEajt+b9PJB5II+Zpqul/QdMtvmCvq3NUg0LVzpOtnZI2bS+0+udT3Mykr+XfzHPNE3K2qa0d8t+l5x8L/q/tYGMmhoA2hnz1fJajcIPa7dFwe3ZTUI/wIdF+gSnbwW60u15pbpyYah+6CsIRNXKkjXNrpDSOQc7DESek= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1720014993; c=relaxed/simple; bh=J9JhzgxVnZi7kUwjZIrWhFBrhcDDsErZrHgSE8TNrHQ=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=WHE1ITsohO02+fQCUs6QbXpUTpa7iGMQrwHPu+3Gge8J6uiwBYWL1YLYfZxQjHLHKyk5UyD8ZvVKKwXzb97B65KsKfefbP9lsYerp5AtNxbJKkPxN9+ccUMHtxUJLDOTouU5UX+64eDiWb+DQP5IYoTUNk/U3Ua+scnPblNqqTg= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=a3+BHr5O; arc=none smtp.client-ip=209.85.216.53 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="a3+BHr5O" Received: by mail-pj1-f53.google.com with SMTP id 98e67ed59e1d1-2c825f0b381so3269196a91.1 for ; Wed, 03 Jul 2024 06:56:32 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1720014991; x=1720619791; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=Dh8siMXxU0DDHgTSZBdEsJyfN+izXvs/CbbfSpXoPos=; b=a3+BHr5OIqIf0NQVe6u/WjbzfXaG2e4bNPsseg6YCCJj46QjiR9drXXzzOkMFg1hIB Hf5JtPe1D9jxqT543YWQ3jvrichtVy1AsR/Ioa4n8A1lzbtyqMmIAqPqb4i0NerX7qkH PrFHaYYMAuUWQUG7S3da2R1klfG3ntG1H0+XOzZ/g4Ddp2HT28xtt7YFTYDimUIsSGJC ocJ3ZS7OGtFK8bYggG8NE5/AMo31HlaNNZ6rLz20wxt3RnwP+eESe7rQTDEhqbIqOc02 oZkLZy9ugLh949mvMbOf9QZzOMKRLi5AaUj6avO3P83Bepu4VREqGFrwZsCeAaBtkdkq eGFg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1720014991; x=1720619791; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=Dh8siMXxU0DDHgTSZBdEsJyfN+izXvs/CbbfSpXoPos=; b=M/vY7DEsaM3vKHXitHwOweBr6bnhKA7uobYdB9zcNb0xshct+h/j8bzhP3iJwpxYay TMTwAm5/zkfJhUXIueMigruuOUC2CClehi5ZW2l9R0YaviNkxFty/ZYQWs90OtJt/pqa VHpViRLWvSfQT6lJiXDhAM0jtzsZfU3bOTDVNaFTMPZ/R6cdf6DX8yKzqDl+l6Zr5EBn ZLL6sg1WRC8gxyVanol+CvAz7D7mF8aZMbBxe2X6mhvIoob/FoA+RqKjmxj7w2hRF3x3 3dxmcdNtzStd+Ky7Xxx4dP9IeOykYbUzIR0KmizoppR00RRFhftIVpUG1j2Oz8MBWqsj fkBw== X-Gm-Message-State: AOJu0Yx9m2npaPx9aIHxJmC+mWG0YZsJkP/Bdz3Khtr5VC2bigF95C0c PQ8LoA5W3RK4Vi1Gw8CSVUKm8JdKnA7T2BulzEMY0+7Q1RegSHF0UY3C8Xo1 X-Google-Smtp-Source: AGHT+IHKercpxulnryj6F6PLRMFT0Cs5kOrf1zaqu5SqLDYy5AO/hwe161lC2HDn0z18NpeqdUn0LA== X-Received: by 2002:a17:90a:ce82:b0:2c9:6c18:4ae5 with SMTP id 98e67ed59e1d1-2c96c185396mr2375180a91.46.1720014990836; Wed, 03 Jul 2024 06:56:30 -0700 (PDT) Received: from localhost ([2605:52c0:1:4cf:6c5a:92ff:fe25:ceff]) by smtp.gmail.com with ESMTPSA id 98e67ed59e1d1-2c91ce6ec6asm10877558a91.26.2024.07.03.06.56.30 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 03 Jul 2024 06:56:30 -0700 (PDT) Date: Wed, 3 Jul 2024 21:56:30 +0800 From: shejialuo To: git@vger.kernel.org Cc: Patrick Steinhardt , Karthik Nayak , Junio C Hamano , Eric Sunshine Subject: [GSoC][PATCH v7 1/9] fsck: rename "skiplist" to "oid_skiplist" Message-ID: References: Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: Because we introduce ref consistency check. The original "skiplist" is a common option which is set up during handling user configs. To avoid causing ambiguity, rename "skiplist" to "oid_skiplist". Mentored-by: Patrick Steinhardt Mentored-by: Karthik Nayak Signed-off-by: shejialuo --- fsck.c | 4 ++-- fsck.h | 2 +- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/fsck.c b/fsck.c index eea7145470..1960bfeba9 100644 --- a/fsck.c +++ b/fsck.c @@ -205,7 +205,7 @@ void fsck_set_msg_types(struct fsck_options *options, const char *values) if (!strcmp(buf, "skiplist")) { if (equal == len) die("skiplist requires a path"); - oidset_parse_file(&options->skiplist, buf + equal + 1, + oidset_parse_file(&options->oid_skiplist, buf + equal + 1, the_repository->hash_algo); buf += len + 1; continue; @@ -223,7 +223,7 @@ void fsck_set_msg_types(struct fsck_options *options, const char *values) static int object_on_skiplist(struct fsck_options *opts, const struct object_id *oid) { - return opts && oid && oidset_contains(&opts->skiplist, oid); + return opts && oid && oidset_contains(&opts->oid_skiplist, oid); } __attribute__((format (printf, 5, 6))) diff --git a/fsck.h b/fsck.h index 6085a384f6..1ee3dd85ba 100644 --- a/fsck.h +++ b/fsck.h @@ -136,7 +136,7 @@ struct fsck_options { fsck_error error_func; unsigned strict:1; enum fsck_msg_type *msg_type; - struct oidset skiplist; + struct oidset oid_skiplist; struct oidset gitmodules_found; struct oidset gitmodules_done; struct oidset gitattributes_found; From patchwork Wed Jul 3 13:56:47 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: shejialuo X-Patchwork-Id: 13722265 Received: from mail-pl1-f175.google.com (mail-pl1-f175.google.com [209.85.214.175]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 139731741CF for ; Wed, 3 Jul 2024 13:56:48 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.175 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1720015010; cv=none; b=D1lFs/6I6S7ygk5AKNxx67tD/SNkp2rHf/rL7/Vm4NQW8xipQCK5hG68oOMWGvYbFxZ0ye+h9pRkN3CZQPXjtJj7ey/Yso3SHxNicsm4h/zXD+ZqM9ftYGgl/E75JjsoqqmSAWb22gLp0n6NcaF1kwnEJlJakAurR0bsN/yE1rI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1720015010; c=relaxed/simple; bh=v8VmFdsty0kB+4InkuHUy7CQZTu3iJ7nCMvoEIJGbhw=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=i+EJrw0mukQR6o7RhoHenWZ3zoG17LpeuwJdoO3vKwCAvgL6m/PXQMQjqN2QzRdHQCrtVbUgbdS1GuXXXUt+ixo0+fXpR+hmOyFqOc07RZl6hDhlgJkZH4ePNR7BUVM8kbOmkBPpDhuFUg6btZMME88eFeaIfIjU9QruytJ/xX0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=MlZGKl9j; arc=none smtp.client-ip=209.85.214.175 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="MlZGKl9j" Received: by mail-pl1-f175.google.com with SMTP id d9443c01a7336-1fab50496f0so33171495ad.2 for ; Wed, 03 Jul 2024 06:56:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1720015008; x=1720619808; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=hM1jUVbFa2gV7nYe65iTj5F6/NS+XiByT6O/hhGxcbM=; b=MlZGKl9jdpVS69+APnmghU+j3/X3qROaUYOCJTbH9Q7nGc2TpZieQ6IyROAESKNtIu oYASOjj7DXYp1EHdAeY9UILc1l/DCNVtSePyKYQ+BRXU2gaUjYpf0KDdJcaZwuYZxWns nHXkP9+Sq4E4KOeknJhLFIW0PctHk3lsVmcyR9q267cUFfajDTkWDyNzbQBg4D6FqbQy Hkmj5OouJyseyH/8d7u9dZxCApQavy+bBpbsP4MvIBIqWvj4papYd6g35ieLlpZQGad5 WpPPftjoo322kvl85SDtHATddV30rWwdFJaCssEqQSBWn7RN/Eie5/KRj0sNaiCWp4gC NMZw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1720015008; x=1720619808; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=hM1jUVbFa2gV7nYe65iTj5F6/NS+XiByT6O/hhGxcbM=; b=KzpeJnClkPPv/KKwqydwBmH0e2w4PqvgukjtVQw123DSa4IPinVCuXtAKOW5rs04s/ C7y8H0sQM9F6tdQbhCh/j01afIHrYazjFl5PfgycJjLl3XBQ1r2CAaLe8kNlacbrteG7 6tJmqGklT4JvJ0wCsBJENZh8f0yGycl1ZfXl9PxXB+aBIzXkmcICM1akDpWf970kMqTS M/0EKNIiYBBoAQFKmfcPvo+o5FxmBhwgb566i9nh+7D5NvFiz80HJ2YWqL3Wn3ikKgw3 O75oz5jCyJ6Im28rTk64NbbQ9waI48ZYOPSSmj33K9KvQP0lvPkA8uNhiMNtg3HaBaMj 8bBQ== X-Gm-Message-State: AOJu0YxX8fwsVUy84A2DVQ0+yBdxStpFxs7xWYCydrZJ5rqM9VZ0RFjp T16TpX/+KiO8v7l88DVGfxI2kDCj64WHmNsq4UKFExQsEfZvVFrd5f2Wrxyy X-Google-Smtp-Source: AGHT+IHQ+rlhLhxUwjC4Rs0NVvJf7VVJvwfOTNbgiWB7x8uu5l1tjYfo3UqWoC99Jt2gMXr+UcmV9Q== X-Received: by 2002:a17:902:e849:b0:1fa:1599:386b with SMTP id d9443c01a7336-1fadbc73e81mr69486845ad.20.1720015007715; Wed, 03 Jul 2024 06:56:47 -0700 (PDT) Received: from localhost ([2605:52c0:1:4cf:6c5a:92ff:fe25:ceff]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-1fac1535d07sm103310585ad.123.2024.07.03.06.56.46 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 03 Jul 2024 06:56:47 -0700 (PDT) Date: Wed, 3 Jul 2024 21:56:47 +0800 From: shejialuo To: git@vger.kernel.org Cc: Patrick Steinhardt , Karthik Nayak , Junio C Hamano , Eric Sunshine Subject: [GSoC][PATCH v7 2/9] fsck: add a unified interface for reporting fsck messages Message-ID: References: Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: The static function "report" provided by "fsck.c" aims at reporting the problems related to object database which cannot be reused for refs. In order to provide a unified interface which can report either objects or refs, create a new function "vfsck_report" by adding "checked_ref_name" parameter following the "report" prototype. However, instead of using "...", provide "va_list" to allow more flexibility. The "vfsck_report" function will use "error_func" registered in "fsck_options" function to report customized messages. Change "error_func" prototype to align with the "vfsck_report". Change "report" function to make it use "vfsck_report" to report objects-related messages. Add a new function called "fsck_refs_report" to use "vfsck_report" to report refs-related messages. Mentored-by: Patrick Steinhardt Mentored-by: Karthik Nayak Signed-off-by: shejialuo --- builtin/fsck.c | 15 ++++----- builtin/mktag.c | 1 + fsck.c | 81 ++++++++++++++++++++++++++++++++++++------------- fsck.h | 42 ++++++++++++++++--------- object-file.c | 11 ++++--- 5 files changed, 102 insertions(+), 48 deletions(-) diff --git a/builtin/fsck.c b/builtin/fsck.c index d13a226c2e..de34538c4f 100644 --- a/builtin/fsck.c +++ b/builtin/fsck.c @@ -89,12 +89,13 @@ static int objerror(struct object *obj, const char *err) return -1; } -static int fsck_error_func(struct fsck_options *o UNUSED, - const struct object_id *oid, - enum object_type object_type, - enum fsck_msg_type msg_type, - enum fsck_msg_id msg_id UNUSED, - const char *message) +static int fsck_objects_error_func(struct fsck_options *o UNUSED, + const struct object_id *oid, + enum object_type object_type, + const char *checked_ref_name UNUSED, + enum fsck_msg_type msg_type, + enum fsck_msg_id msg_id UNUSED, + const char *message) { switch (msg_type) { case FSCK_WARN: @@ -938,7 +939,7 @@ int cmd_fsck(int argc, const char **argv, const char *prefix) fsck_walk_options.walk = mark_object; fsck_obj_options.walk = mark_used; - fsck_obj_options.error_func = fsck_error_func; + fsck_obj_options.error_func = fsck_objects_error_func; if (check_strict) fsck_obj_options.strict = 1; diff --git a/builtin/mktag.c b/builtin/mktag.c index 4767f1a97e..42f945c584 100644 --- a/builtin/mktag.c +++ b/builtin/mktag.c @@ -20,6 +20,7 @@ static struct fsck_options fsck_options = FSCK_OPTIONS_STRICT; static int mktag_fsck_error_func(struct fsck_options *o UNUSED, const struct object_id *oid UNUSED, enum object_type object_type UNUSED, + const char *checked_ref_name UNUSED, enum fsck_msg_type msg_type, enum fsck_msg_id msg_id UNUSED, const char *message) diff --git a/fsck.c b/fsck.c index 1960bfeba9..7182ce8e80 100644 --- a/fsck.c +++ b/fsck.c @@ -226,12 +226,18 @@ static int object_on_skiplist(struct fsck_options *opts, return opts && oid && oidset_contains(&opts->oid_skiplist, oid); } -__attribute__((format (printf, 5, 6))) -static int report(struct fsck_options *options, - const struct object_id *oid, enum object_type object_type, - enum fsck_msg_id msg_id, const char *fmt, ...) +/* + * Provide a unified interface for either fscking refs or objects. + * It will get the current msg error type and call the error_func callback + * which is registered in the "fsck_options" struct. + */ +static int vfsck_report(struct fsck_options *options, + const struct object_id *oid, + enum object_type object_type, + const char *checked_ref_name, + enum fsck_msg_id msg_id, const char *fmt, va_list ap) { - va_list ap; + va_list ap_copy; struct strbuf sb = STRBUF_INIT; enum fsck_msg_type msg_type = fsck_msg_type(msg_id, options); int result; @@ -250,9 +256,9 @@ static int report(struct fsck_options *options, prepare_msg_ids(); strbuf_addf(&sb, "%s: ", msg_id_info[msg_id].camelcased); - va_start(ap, fmt); - strbuf_vaddf(&sb, fmt, ap); - result = options->error_func(options, oid, object_type, + va_copy(ap_copy, ap); + strbuf_vaddf(&sb, fmt, ap_copy); + result = options->error_func(options, oid, object_type, checked_ref_name, msg_type, msg_id, sb.buf); strbuf_release(&sb); va_end(ap); @@ -260,6 +266,36 @@ static int report(struct fsck_options *options, return result; } +__attribute__((format (printf, 5, 6))) +static int report(struct fsck_options *options, + const struct object_id *oid, enum object_type object_type, + enum fsck_msg_id msg_id, const char *fmt, ...) +{ + va_list ap; + int result; + va_start(ap, fmt); + result = vfsck_report(options, oid, object_type, NULL, + msg_id, fmt, ap); + va_end(ap); + return result; +} + + + +int fsck_refs_report(struct fsck_options *options, + const struct object_id *oid, + const char *checked_ref_name, + enum fsck_msg_id msg_id, const char *fmt, ...) +{ + va_list ap; + int result; + va_start(ap, fmt); + result = vfsck_report(options, oid, OBJ_NONE, + checked_ref_name, msg_id, fmt, ap); + va_end(ap); + return result; +} + void fsck_enable_object_names(struct fsck_options *options) { if (!options->object_names) @@ -1200,12 +1236,13 @@ int fsck_buffer(const struct object_id *oid, enum object_type type, type); } -int fsck_error_function(struct fsck_options *o, - const struct object_id *oid, - enum object_type object_type UNUSED, - enum fsck_msg_type msg_type, - enum fsck_msg_id msg_id UNUSED, - const char *message) +int fsck_objects_error_function(struct fsck_options *o, + const struct object_id *oid, + enum object_type object_type UNUSED, + const char *checked_ref_name UNUSED, + enum fsck_msg_type msg_type, + enum fsck_msg_id msg_id UNUSED, + const char *message) { if (msg_type == FSCK_WARN) { warning("object %s: %s", fsck_describe_object(o, oid), message); @@ -1303,16 +1340,18 @@ int git_fsck_config(const char *var, const char *value, * Custom error callbacks that are used in more than one place. */ -int fsck_error_cb_print_missing_gitmodules(struct fsck_options *o, - const struct object_id *oid, - enum object_type object_type, - enum fsck_msg_type msg_type, - enum fsck_msg_id msg_id, - const char *message) +int fsck_objects_error_cb_print_missing_gitmodules(struct fsck_options *o, + const struct object_id *oid, + enum object_type object_type, + const char *checked_ref_name, + enum fsck_msg_type msg_type, + enum fsck_msg_id msg_id, + const char *message) { if (msg_id == FSCK_MSG_GITMODULES_MISSING) { puts(oid_to_hex(oid)); return 0; } - return fsck_error_function(o, oid, object_type, msg_type, msg_id, message); + return fsck_objects_error_function(o, oid, object_type, checked_ref_name, + msg_type, msg_id, message); } diff --git a/fsck.h b/fsck.h index 1ee3dd85ba..f703dfb5e8 100644 --- a/fsck.h +++ b/fsck.h @@ -114,22 +114,27 @@ int is_valid_msg_type(const char *msg_id, const char *msg_type); typedef int (*fsck_walk_func)(struct object *obj, enum object_type object_type, void *data, struct fsck_options *options); -/* callback for fsck_object, type is FSCK_ERROR or FSCK_WARN */ +/* + * callback function for reporting errors when checking either objects or refs + */ typedef int (*fsck_error)(struct fsck_options *o, const struct object_id *oid, enum object_type object_type, + const char *checked_ref_name, enum fsck_msg_type msg_type, enum fsck_msg_id msg_id, const char *message); -int fsck_error_function(struct fsck_options *o, - const struct object_id *oid, enum object_type object_type, - enum fsck_msg_type msg_type, enum fsck_msg_id msg_id, - const char *message); -int fsck_error_cb_print_missing_gitmodules(struct fsck_options *o, - const struct object_id *oid, - enum object_type object_type, - enum fsck_msg_type msg_type, - enum fsck_msg_id msg_id, - const char *message); +int fsck_objects_error_function(struct fsck_options *o, + const struct object_id *oid, enum object_type object_type, + const char *checked_ref_name, + enum fsck_msg_type msg_type, enum fsck_msg_id msg_id, + const char *message); +int fsck_objects_error_cb_print_missing_gitmodules(struct fsck_options *o, + const struct object_id *oid, + enum object_type object_type, + const char *checked_ref_name, + enum fsck_msg_type msg_type, + enum fsck_msg_id msg_id, + const char *message); struct fsck_options { fsck_walk_func walk; @@ -145,12 +150,12 @@ struct fsck_options { }; #define FSCK_OPTIONS_DEFAULT { \ - .skiplist = OIDSET_INIT, \ + .oid_skiplist = OIDSET_INIT, \ .gitmodules_found = OIDSET_INIT, \ .gitmodules_done = OIDSET_INIT, \ .gitattributes_found = OIDSET_INIT, \ .gitattributes_done = OIDSET_INIT, \ - .error_func = fsck_error_function \ + .error_func = fsck_objects_error_function \ } #define FSCK_OPTIONS_STRICT { \ .strict = 1, \ @@ -158,7 +163,7 @@ struct fsck_options { .gitmodules_done = OIDSET_INIT, \ .gitattributes_found = OIDSET_INIT, \ .gitattributes_done = OIDSET_INIT, \ - .error_func = fsck_error_function, \ + .error_func = fsck_objects_error_function, \ } #define FSCK_OPTIONS_MISSING_GITMODULES { \ .strict = 1, \ @@ -166,7 +171,7 @@ struct fsck_options { .gitmodules_done = OIDSET_INIT, \ .gitattributes_found = OIDSET_INIT, \ .gitattributes_done = OIDSET_INIT, \ - .error_func = fsck_error_cb_print_missing_gitmodules, \ + .error_func = fsck_objects_error_cb_print_missing_gitmodules, \ } /* descend in all linked child objects @@ -209,6 +214,13 @@ int fsck_tag_standalone(const struct object_id *oid, const char *buffer, */ int fsck_finish(struct fsck_options *options); +__attribute__((format (printf, 5, 6))) +int fsck_refs_report(struct fsck_options *options, + const struct object_id *oid, + const char *checked_ref_name, + enum fsck_msg_id msg_id, + const char *fmt, ...); + /* * Subsystem for storing human-readable names for each object. * diff --git a/object-file.c b/object-file.c index 065103be3e..d2c6427935 100644 --- a/object-file.c +++ b/object-file.c @@ -2470,11 +2470,12 @@ int repo_has_object_file(struct repository *r, * give more context. */ static int hash_format_check_report(struct fsck_options *opts UNUSED, - const struct object_id *oid UNUSED, - enum object_type object_type UNUSED, - enum fsck_msg_type msg_type UNUSED, - enum fsck_msg_id msg_id UNUSED, - const char *message) + const struct object_id *oid UNUSED, + enum object_type object_type UNUSED, + const char *ref_checked_name UNUSED, + enum fsck_msg_type msg_type UNUSED, + enum fsck_msg_id msg_id UNUSED, + const char *message) { error(_("object fails fsck: %s"), message); return 1; From patchwork Wed Jul 3 13:57:02 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: shejialuo X-Patchwork-Id: 13722266 Received: from mail-pl1-f169.google.com (mail-pl1-f169.google.com [209.85.214.169]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 271AC85624 for ; Wed, 3 Jul 2024 13:57:03 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.169 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1720015025; cv=none; b=JG6B+oWtiLA9xSQwTOeGPkaVXv8pZM9xEwedyECtJQMyrdFkmINNMIXLpvMz0Fo8MmlD3bc3Luu3SvNCJId2JyLw5vzArlcuzBYXKfnsblIRlMugma1YzToaTf4Q0Mr+sLwVJ+RMr9YgwFPjtE1U+DL+NTKJdFuxEyPESpsY8ns= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1720015025; c=relaxed/simple; bh=jQghWynB7l/myLbFQEpmq/CunycbaK16wPnKbUtPY+Q=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=DGgQQbmI9bVCZycFGVPqlj9xhlBA+GTmqVUHv4Umt5TF4IXg5s6htWmMi2a5Lt3NforD0xmjKUQj0ZIiO/Q+JY8ACieCXIAotfzqj5uauvWzF9SEDmWFiL1yyiuE84clwHWpfsP9ojQbIVozKIyazRMHnv7ipO93byvggcpMtJw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=PhhUROyi; arc=none smtp.client-ip=209.85.214.169 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="PhhUROyi" Received: by mail-pl1-f169.google.com with SMTP id d9443c01a7336-1fb222a8eaeso2823105ad.3 for ; Wed, 03 Jul 2024 06:57:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1720015023; x=1720619823; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=wcHdesL+nRhq/wtl9I6I0uiecL39HaGJC1i/WB2Rjzk=; b=PhhUROyi/g+fl1WML9EtrQWSSx3U14+Uu7MlaflyWFAbIfukj8L8vA0eiZ6g2d3eEv fB1EiM120FqyetDTHQj5gGrFrq5+DrLX8K7uhK94Cjz+vFNLKkKWoozdH3cumMT8sNnu nOEFYsMmG7quHAe4ItC7tUL0/bvHTUBOeEdzS3Btez6Nx5VRq2UG7eFALHHN3Hot0WCo yDIe0QsNQQ+FWB/KUOUUx1GjPIQSWtpcc+2A9r7bVgMDvjusuXULa/UeJNl2vKUp/94n LoBhdgkYm0i2kAROqgM0jCNDbe3VK9VvIEZniHLik8KP1DHkSSkQeJZ7kbLtwUg9hddr G6jw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1720015023; x=1720619823; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=wcHdesL+nRhq/wtl9I6I0uiecL39HaGJC1i/WB2Rjzk=; b=rSKGIS+2SeLmhdNz7wOFghvSnH/D9iRPmSyzQsnQZ3CXZeJoyLWOedawun/8IewPhi Xsc7ZBAX3kg/ohE54SD2YdcFDr4WmvcIHvWIMSJDnb/jEnaJEZDuiPOI7Xs3K4pipGZY TF717o99u+/fDCbSaifsZW78fn+ptyocssIuT3n/czplHzfI2dpx3wFboPhD7fSGTXkg f+U0tYnR5N8OjHXfPWXSsiu0xDrz0LhcEifdEvJ5xevmPGBYBu1Bw4ynEdonWViax5pE MS8xSFpCHsFv6UFicJqyEjDAnLOZ3AEMkHF/O1m1d1A34lvmAUR63rhNDDtWUVYQvGcQ nGzQ== X-Gm-Message-State: AOJu0Ywi+1Ip3Mku2NEezstjC0gEiMX5S+o7umIwoNQ3eH508JyZ/scI U11ZWxGG9wIpYIRm+ARKMQbl04cE41L7G+ig212SevG5/B1l4hJWtfa+KlXt X-Google-Smtp-Source: AGHT+IHWI0go9aDr3Wbvb60ZjER4Dy3LG1+IyvR9pNQyTy+4r7kn/Pq8HgBgO4V3nsk2gxxeyMastQ== X-Received: by 2002:a17:903:41d0:b0:1fb:167e:fb0f with SMTP id d9443c01a7336-1fb167efe72mr12669985ad.18.1720015022923; Wed, 03 Jul 2024 06:57:02 -0700 (PDT) Received: from localhost ([2605:52c0:1:4cf:6c5a:92ff:fe25:ceff]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-1fac1535b1esm103922435ad.140.2024.07.03.06.57.01 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 03 Jul 2024 06:57:02 -0700 (PDT) Date: Wed, 3 Jul 2024 21:57:02 +0800 From: shejialuo To: git@vger.kernel.org Cc: Patrick Steinhardt , Karthik Nayak , Junio C Hamano , Eric Sunshine Subject: [GSoC][PATCH v7 3/9] fsck: add refs-related options and error report function Message-ID: References: Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: Add refs-related options to the "fsck_options", create refs-specific "error_func" callback "fsck_refs_error_function". "fsck_refs_error_function" will use the "oid" parameter. When the caller passes the oid, it will use "oid_to_hex" to get the corresponding hex value to report to the caller. Last, add "FSCK_REFS_OPTIONS_DEFAULT" and "FSCK_REFS_OPTIONS_STRICT" macros to create refs options easily. Mentored-by: Patrick Steinhardt Mentored-by: Karthik Nayak Signed-off-by: shejialuo --- fsck.c | 22 ++++++++++++++++++++++ fsck.h | 15 +++++++++++++++ 2 files changed, 37 insertions(+) diff --git a/fsck.c b/fsck.c index 7182ce8e80..d1dcbdcac2 100644 --- a/fsck.c +++ b/fsck.c @@ -1252,6 +1252,28 @@ int fsck_objects_error_function(struct fsck_options *o, return 1; } +int fsck_refs_error_function(struct fsck_options *options UNUSED, + const struct object_id *oid, + enum object_type object_type UNUSED, + const char *checked_ref_name, + enum fsck_msg_type msg_type, + enum fsck_msg_id msg_id UNUSED, + const char *message) +{ + static struct strbuf sb = STRBUF_INIT; + + strbuf_addstr(&sb, checked_ref_name); + if (oid) + strbuf_addf(&sb, " -> (%s)", oid_to_hex(oid)); + + if (msg_type == FSCK_WARN) { + warning("%s: %s", sb.buf, message); + return 0; + } + error("%s: %s", sb.buf, message); + return 1; +} + static int fsck_blobs(struct oidset *blobs_found, struct oidset *blobs_done, enum fsck_msg_id msg_missing, enum fsck_msg_id msg_type, struct fsck_options *options, const char *blob_type) diff --git a/fsck.h b/fsck.h index f703dfb5e8..246055c0f9 100644 --- a/fsck.h +++ b/fsck.h @@ -135,11 +135,19 @@ int fsck_objects_error_cb_print_missing_gitmodules(struct fsck_options *o, enum fsck_msg_type msg_type, enum fsck_msg_id msg_id, const char *message); +int fsck_refs_error_function(struct fsck_options *options, + const struct object_id *oid, + enum object_type object_type, + const char *checked_ref_name, + enum fsck_msg_type msg_type, + enum fsck_msg_id msg_id, + const char *message); struct fsck_options { fsck_walk_func walk; fsck_error error_func; unsigned strict:1; + unsigned verbose_refs:1; enum fsck_msg_type *msg_type; struct oidset oid_skiplist; struct oidset gitmodules_found; @@ -173,6 +181,13 @@ struct fsck_options { .gitattributes_done = OIDSET_INIT, \ .error_func = fsck_objects_error_cb_print_missing_gitmodules, \ } +#define FSCK_REFS_OPTIONS_DEFAULT { \ + .error_func = fsck_refs_error_function, \ +} +#define FSCK_REFS_OPTIONS_STRICT { \ + .strict = 1, \ + .error_func = fsck_refs_error_function, \ +} /* descend in all linked child objects * the return value is: From patchwork Wed Jul 3 13:57:51 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: shejialuo X-Patchwork-Id: 13722267 Received: from mail-pf1-f176.google.com (mail-pf1-f176.google.com [209.85.210.176]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 0699F17A599 for ; Wed, 3 Jul 2024 13:57:52 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.176 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1720015074; cv=none; b=IZkwEPc460lpEkGyavhyN0afpS2KaMOiCjpPuGoUAhUWk0NvUzjs36h0fOEshguDiQmzl6hFz4W4zu5CKuTvxItlndV4PjElCC03EM9uI+gqEQgHHdOHBDe9f71v0GE0Dx6jc9P7RZFZxwzDomDYEf9bw5XeU3DJfXLCXS6T4+k= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1720015074; c=relaxed/simple; bh=UrDMzI/fDR/GyLZnZt8IH1yAlAowxGIlDTM2eXKL51A=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=L9hiP70EFkRqfW0sfOSTjulfQgJ74fE+SWeCKU6t1Pntxq46FnuTTkuaRAo/tN8uTDIUmf01pvArF8UgkuK3jyKO+sdAvetvV5aAy0kvSl/8gdsGsuj/PtV7ZdtoEQnLnh5rLlTk5PtvLyjA+UXqHL0JqBU0XyufH6pTmMuPKCU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=HsvofsiM; arc=none smtp.client-ip=209.85.210.176 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="HsvofsiM" Received: by mail-pf1-f176.google.com with SMTP id d2e1a72fcca58-70af4868d3dso268578b3a.3 for ; Wed, 03 Jul 2024 06:57:52 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1720015072; x=1720619872; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=tRS82vf6Xog1qs7YecMOL10IUWpS6A53mOBHrDZUGGM=; b=HsvofsiMZ0O3ihdmXZ+NWYhXrwjkhiZa6DFOXz3VZ1ORWMCcprMk/jKQNhgHRXCgjP pvF4TWIA+sPnHkBEn4KavsDoJ/XdBZcE42LyGIw9lWS9ntNZPrVb2JTh0UZWnbhtvDWl uAPQWxSoz73WbU+eBOqwj75sAQX+8LJY0zp6M2m2uChd+IsejqWGhFSC7TRXByShnjs9 jXpsQVsC/No6346vRwS1XYY+CEIuXyq9//h/hG2xec80J+eI7pvQ8kXMwjpkWB4XOCmt jYkeFFpFk3IKGpIHSBApVXUY5BSr0rCbubrfTSrc1sQ8at88Ch/bQYZXd2V+FigHUdYk pWEw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1720015072; x=1720619872; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=tRS82vf6Xog1qs7YecMOL10IUWpS6A53mOBHrDZUGGM=; b=ptCr5RGrQ5Yx03xhCbK5xwX2OBg7z++Wq9hfelucpmdXu3pLXdlow93mUV4Tq6NO16 90HbJVrSVGl765D6lso+rXR2pvNI4yYVtPi12viEsG7dhjU9rUilavEjf4Jz9vDR0yNh PVmcHpNqKw5C/KE5qShI1Bf7E2tzwQY53RGJcEXylEBZfPOiL/UD0TCIPjT/agDtiHSF DNo6HLRjLT3G+/g/rTUOzlb2PUIXkrp2RoXv3NTkSCTgdE0yaOtZ6q+bF1b3BSqiASt1 ndMv3mXdt2KLmMsM0TvPxDNxyTIFOkexWWiWVoISTbDUlw+9KKW/olSSQugDI+k3xyqt x2/g== X-Gm-Message-State: AOJu0YxBSajqQ872hHTIs7ZjgiqxjVACPtbl+bYFGRJ8wjcDKD58yneR 0FD9jcg1Uv4TpXgr9+reJGHLb3ntNoL3/lcV48rO7mSRb3bXhGIZUERR2KOd X-Google-Smtp-Source: AGHT+IHzm3RUn7qewT6ZevdLWmvidfNTxaLVCR8dDYFR05FcWaIaDmpwowZ/s8+8JAfmHjORuaw4Bg== X-Received: by 2002:a05:6a20:d510:b0:1bf:5ca2:a9b6 with SMTP id adf61e73a8af0-1bf5ca2ac18mr8036987637.50.1720015071406; Wed, 03 Jul 2024 06:57:51 -0700 (PDT) Received: from localhost ([2605:52c0:1:4cf:6c5a:92ff:fe25:ceff]) by smtp.gmail.com with ESMTPSA id 41be03b00d2f7-72c6c8ec50csm7725756a12.59.2024.07.03.06.57.50 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 03 Jul 2024 06:57:50 -0700 (PDT) Date: Wed, 3 Jul 2024 21:57:51 +0800 From: shejialuo To: git@vger.kernel.org Cc: Patrick Steinhardt , Karthik Nayak , Junio C Hamano , Eric Sunshine Subject: [GSoC][PATCH v7 4/9] refs: set up ref consistency check infrastructure Message-ID: References: Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: The interfaces defined in the `ref_storage_be` are carefully structured in semantic. It's organized as the five parts: 1. The name and the initialization interfaces. 2. The ref transaction interfaces. 3. The ref internal interfaces (pack, rename and copy). 4. The ref filesystem interfaces. 5. The reflog related interfaces. To keep consistent with the git-fsck(1), add a new interface named "fsck_refs_fn" to the end of "ref_storage_be". This semantic cannot be grouped into any above five categories. Explicitly add blank line to make it different from others. Mentored-by: Patrick Steinhardt Mentored-by: Karthik Nayak Signed-off-by: shejialuo --- refs.c | 5 +++++ refs.h | 8 ++++++++ refs/debug.c | 11 +++++++++++ refs/files-backend.c | 15 ++++++++++++++- refs/packed-backend.c | 8 ++++++++ refs/refs-internal.h | 6 ++++++ refs/reftable-backend.c | 8 ++++++++ 7 files changed, 60 insertions(+), 1 deletion(-) diff --git a/refs.c b/refs.c index bb90a18875..410919246b 100644 --- a/refs.c +++ b/refs.c @@ -318,6 +318,11 @@ int check_refname_format(const char *refname, int flags) return check_or_sanitize_refname(refname, flags, NULL); } +int refs_fsck(struct ref_store *refs, struct fsck_options *o) +{ + return refs->be->fsck(refs, o); +} + void sanitize_refname_component(const char *refname, struct strbuf *out) { if (check_or_sanitize_refname(refname, REFNAME_ALLOW_ONELEVEL, out)) diff --git a/refs.h b/refs.h index 0ecba21b4a..804d6a7fce 100644 --- a/refs.h +++ b/refs.h @@ -4,6 +4,7 @@ #include "commit.h" #include "repository.h" +struct fsck_options; struct object_id; struct ref_store; struct strbuf; @@ -541,6 +542,13 @@ int refs_for_each_reflog(struct ref_store *refs, each_reflog_fn fn, void *cb_dat */ int check_refname_format(const char *refname, int flags); +/* + * Check the reference database for consistency. Return 0 if refs and + * reflogs are consistent, and non-zero otherwise. The errors will be + * written to stderr. + */ +int refs_fsck(struct ref_store *refs, struct fsck_options *o); + /* * Apply the rules from check_refname_format, but mutate the result until it * is acceptable, and place the result in "out". diff --git a/refs/debug.c b/refs/debug.c index 547d9245b9..45e2e784a0 100644 --- a/refs/debug.c +++ b/refs/debug.c @@ -419,6 +419,15 @@ static int debug_reflog_expire(struct ref_store *ref_store, const char *refname, return res; } +static int debug_fsck(struct ref_store *ref_store, + struct fsck_options *o) +{ + struct debug_ref_store *drefs = (struct debug_ref_store *)ref_store; + int res = drefs->refs->be->fsck(drefs->refs, o); + trace_printf_key(&trace_refs, "fsck: %d\n", res); + return res; +} + struct ref_storage_be refs_be_debug = { .name = "debug", .init = NULL, @@ -451,4 +460,6 @@ struct ref_storage_be refs_be_debug = { .create_reflog = debug_create_reflog, .delete_reflog = debug_delete_reflog, .reflog_expire = debug_reflog_expire, + + .fsck = debug_fsck, }; diff --git a/refs/files-backend.c b/refs/files-backend.c index aa52d9be7c..d89eeda8ef 100644 --- a/refs/files-backend.c +++ b/refs/files-backend.c @@ -3408,6 +3408,17 @@ static int files_ref_store_remove_on_disk(struct ref_store *ref_store, return ret; } +static int files_fsck(struct ref_store *ref_store, + struct fsck_options *o) +{ + int ret; + struct files_ref_store *refs = + files_downcast(ref_store, REF_STORE_READ, "fsck"); + + ret = refs->packed_ref_store->be->fsck(refs->packed_ref_store, o); + return ret; +} + struct ref_storage_be refs_be_files = { .name = "files", .init = files_ref_store_init, @@ -3434,5 +3445,7 @@ struct ref_storage_be refs_be_files = { .reflog_exists = files_reflog_exists, .create_reflog = files_create_reflog, .delete_reflog = files_delete_reflog, - .reflog_expire = files_reflog_expire + .reflog_expire = files_reflog_expire, + + .fsck = files_fsck, }; diff --git a/refs/packed-backend.c b/refs/packed-backend.c index a0666407cd..5209b0b212 100644 --- a/refs/packed-backend.c +++ b/refs/packed-backend.c @@ -1735,6 +1735,12 @@ static struct ref_iterator *packed_reflog_iterator_begin(struct ref_store *ref_s return empty_ref_iterator_begin(); } +static int packed_fsck(struct ref_store *ref_store, + struct fsck_options *o) +{ + return 0; +} + struct ref_storage_be refs_be_packed = { .name = "packed", .init = packed_ref_store_init, @@ -1762,4 +1768,6 @@ struct ref_storage_be refs_be_packed = { .create_reflog = NULL, .delete_reflog = NULL, .reflog_expire = NULL, + + .fsck = packed_fsck, }; diff --git a/refs/refs-internal.h b/refs/refs-internal.h index fa975d69aa..a905e187cd 100644 --- a/refs/refs-internal.h +++ b/refs/refs-internal.h @@ -4,6 +4,7 @@ #include "refs.h" #include "iterator.h" +struct fsck_options; struct ref_transaction; /* @@ -650,6 +651,9 @@ typedef int read_raw_ref_fn(struct ref_store *ref_store, const char *refname, typedef int read_symbolic_ref_fn(struct ref_store *ref_store, const char *refname, struct strbuf *referent); +typedef int fsck_fn(struct ref_store *ref_store, + struct fsck_options *o); + struct ref_storage_be { const char *name; ref_store_init_fn *init; @@ -677,6 +681,8 @@ struct ref_storage_be { create_reflog_fn *create_reflog; delete_reflog_fn *delete_reflog; reflog_expire_fn *reflog_expire; + + fsck_fn *fsck; }; extern struct ref_storage_be refs_be_files; diff --git a/refs/reftable-backend.c b/refs/reftable-backend.c index fbe74c239d..b5a1a526df 100644 --- a/refs/reftable-backend.c +++ b/refs/reftable-backend.c @@ -2303,6 +2303,12 @@ static int reftable_be_reflog_expire(struct ref_store *ref_store, return ret; } +static int reftable_be_fsck(struct ref_store *ref_store, + struct fsck_options *o) +{ + return 0; +} + struct ref_storage_be refs_be_reftable = { .name = "reftable", .init = reftable_be_init, @@ -2330,4 +2336,6 @@ struct ref_storage_be refs_be_reftable = { .create_reflog = reftable_be_create_reflog, .delete_reflog = reftable_be_delete_reflog, .reflog_expire = reftable_be_reflog_expire, + + .fsck = reftable_be_fsck, }; From patchwork Wed Jul 3 13:58:06 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: shejialuo X-Patchwork-Id: 13722268 Received: from mail-pg1-f179.google.com (mail-pg1-f179.google.com [209.85.215.179]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 9593817B435 for ; Wed, 3 Jul 2024 13:58:07 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.215.179 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1720015089; cv=none; b=ZMCOzGEYM8gLZvMxQULbeQMt324fSD26KpprRUNEpzeO68h2q6K08FZ4aWyBSbWs2/CvW2841jwCcbVnJ7P1c3OR/SscnnBp5M/bXaI0mx18sWtSVwwOcafaYimSwmEmVzWhDm8JdZfN/aPz7WatLESzP7VFyie9wJOp3nhN7+o= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1720015089; c=relaxed/simple; bh=D6ETxbtTkMy/f2wA+nHwj2LARo1b8hVS/P1KC22Z0H0=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=fMKbKxYSci1XdtsAQbKKLHv5aiaW8WlIcOhxHwzRXZ4SOqNhkLy8ykuHlJC0QQFja3gPXBpPOz0UYwZxAlH/7IytAJyVSOnvrwowxnvmNXCowXX7zLoEgyknfadOrsWTVXl6GyMHcsVBy+ZXRUx+u9u+DTABnKZkLcQGDpt0rig= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=SJYdpCkE; arc=none smtp.client-ip=209.85.215.179 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="SJYdpCkE" Received: by mail-pg1-f179.google.com with SMTP id 41be03b00d2f7-7515437ff16so1631018a12.2 for ; Wed, 03 Jul 2024 06:58:07 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1720015086; x=1720619886; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=Om4m+E104fYydmOLrPhJ06tXqIyGWOces33eHq6kRRc=; b=SJYdpCkE/9z2YKCFDCYCFvNB3dVG7uSObwy8nS2FTLqP12bYEtUD07N/JWG9QkjyYH yeiP7ipG972yG0TtYo/mzaooeeVv69r3Sg+BaXk4GlcR9eWPjkF87NRACQQAo2bu7M+q URdR07BvbUwnDjPdUCT4lY0YSmCzWsL/F4BeODACGlo1z1RQtOzo+KbWDvdcoyWw4p6j IO1WXVXFtOFI64wfkzBmQpJHxc4JVdd5EWCBslu3IQ62vcgNCkG6Dmy55lkgdG5tkLnh 4qtG7xCqj4RU+EogI3N8oYWP1/YnPLtpn/3noq//o3povu3Fmm8i6LDbkq1NI/mtDQR/ nIFQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1720015086; x=1720619886; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=Om4m+E104fYydmOLrPhJ06tXqIyGWOces33eHq6kRRc=; b=nDF2H9Nl3fW/xbu610tRy1EvQUlCkIinxHj+W50O/OHu6bqVFL9Gdgiyu5KIU7++3t RSPvrqqUDC1b9uq19dUbBfhMtchbycAy04riZBfLCv5T7YbyvYNNRKJ88X59u3iiJiQJ rnmDqik9LToi7D1bpBfx4pVzo9RHZSVPsaUXIH6hNvU4VTIs6AyMJ2DKoKqC7HeGBsyv pms2MAKxQLEvEdL9ECZcEtFSIqFXJQVTpC1iYpLqRaCQQs/rRUXItIoGbtS/PvSZJ0VL 0kmKUaVqkHarrLpKV/qkSR24Q1Hj/ysUBsWVeaip57m5HsYla0DlooOJUuYb3CHCDGI0 IT+Q== X-Gm-Message-State: AOJu0YwAtYyqJ2NI33VhA2niZAsQdLMM46/qwvbeHufsvKBqUEeu4ARd PdbME9nSn3QzUaEYH2iT/KtCyMwIymYv+n1VdCTaPl0ueEXtp2k07GbftN9K X-Google-Smtp-Source: AGHT+IGxtBi6LqkNF+9V/VjwXa52g0BCyusJ7yVFDuqoaTyIsU7pN0CrVPv3MPh0nuIEyQklhBhpaw== X-Received: by 2002:a05:6a20:3a99:b0:1bd:251a:b657 with SMTP id adf61e73a8af0-1bef6241910mr12580904637.52.1720015086378; Wed, 03 Jul 2024 06:58:06 -0700 (PDT) Received: from localhost ([2605:52c0:1:4cf:6c5a:92ff:fe25:ceff]) by smtp.gmail.com with ESMTPSA id 98e67ed59e1d1-2c96f76b09fsm1951485a91.16.2024.07.03.06.58.05 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 03 Jul 2024 06:58:05 -0700 (PDT) Date: Wed, 3 Jul 2024 21:58:06 +0800 From: shejialuo To: git@vger.kernel.org Cc: Patrick Steinhardt , Karthik Nayak , Junio C Hamano , Eric Sunshine Subject: [GSoC][PATCH v7 5/9] builtin/refs: add verify subcommand Message-ID: References: Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: Introduce a new subcommand "verify" in git-refs(1) to allow the user to check the reference database consistency. Mentored-by: Patrick Steinhardt Mentored-by: Karthik Nayak Signed-off-by: shejialuo --- Documentation/git-refs.txt | 13 +++++++++++ builtin/refs.c | 44 ++++++++++++++++++++++++++++++++++++++ 2 files changed, 57 insertions(+) diff --git a/Documentation/git-refs.txt b/Documentation/git-refs.txt index 5b99e04385..1244a85b64 100644 --- a/Documentation/git-refs.txt +++ b/Documentation/git-refs.txt @@ -10,6 +10,7 @@ SYNOPSIS -------- [verse] 'git refs migrate' --ref-format= [--dry-run] +'git refs verify' [--strict] [--verbose] DESCRIPTION ----------- @@ -22,6 +23,9 @@ COMMANDS migrate:: Migrate ref store between different formats. +verify:: + Verify reference database consistency. + OPTIONS ------- @@ -39,6 +43,15 @@ include::ref-storage-format.txt[] can be used to double check that the migration works as expected before performing the actual migration. +The following options are specific to 'git refs verify': + +--strict:: + Enable more strict checking, every WARN severity for the `Fsck Messages` + be seen as ERROR. See linkgit:git-fsck[1]. + +--verbose:: + When verifying the reference database consistency, be chatty. + KNOWN LIMITATIONS ----------------- diff --git a/builtin/refs.c b/builtin/refs.c index 46dcd150d4..74720f5e0d 100644 --- a/builtin/refs.c +++ b/builtin/refs.c @@ -1,4 +1,6 @@ #include "builtin.h" +#include "config.h" +#include "fsck.h" #include "parse-options.h" #include "refs.h" #include "repository.h" @@ -7,6 +9,9 @@ #define REFS_MIGRATE_USAGE \ N_("git refs migrate --ref-format= [--dry-run]") +#define REFS_VERIFY_USAGE \ + N_("git refs verify [--strict] [--verbose]") + static int cmd_refs_migrate(int argc, const char **argv, const char *prefix) { const char * const migrate_usage[] = { @@ -58,15 +63,54 @@ static int cmd_refs_migrate(int argc, const char **argv, const char *prefix) return err; } +static int cmd_refs_verify(int argc, const char **argv, const char *prefix) +{ + struct fsck_options fsck_refs_options = FSCK_REFS_OPTIONS_DEFAULT; + const char * const verify_usage[] = { + REFS_VERIFY_USAGE, + NULL, + }; + unsigned int verbose = 0, strict = 0; + struct option options[] = { + OPT__VERBOSE(&verbose, N_("be verbose")), + OPT_BOOL(0, "strict", &strict, N_("enable strict checking")), + OPT_END(), + }; + int ret = 0; + + argc = parse_options(argc, argv, prefix, options, verify_usage, 0); + if (argc) + usage(_("too many arguments")); + + if (verbose) + fsck_refs_options.verbose_refs = 1; + if (strict) + fsck_refs_options.strict = 1; + + git_config(git_fsck_config, &fsck_refs_options); + prepare_repo_settings(the_repository); + + ret = refs_fsck(get_main_ref_store(the_repository), &fsck_refs_options); + + /* + * Explicitly free the allocated array and "oid_skiplist" + */ + free(fsck_refs_options.msg_type); + oidset_clear(&fsck_refs_options.oid_skiplist); + return ret; +} + int cmd_refs(int argc, const char **argv, const char *prefix) { const char * const refs_usage[] = { REFS_MIGRATE_USAGE, + REFS_VERIFY_USAGE, NULL, }; parse_opt_subcommand_fn *fn = NULL; struct option opts[] = { OPT_SUBCOMMAND("migrate", &fn, cmd_refs_migrate), + OPT_SUBCOMMAND("verify", &fn, cmd_refs_verify), OPT_END(), }; From patchwork Wed Jul 3 13:58:19 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: shejialuo X-Patchwork-Id: 13722269 Received: from mail-pl1-f169.google.com (mail-pl1-f169.google.com [209.85.214.169]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id DC4B01E869 for ; Wed, 3 Jul 2024 13:58:20 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.169 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1720015102; cv=none; b=QAa/LQdp/1qgOcO7x6X8zYqtuKPnUxzamiHXyAdnzFM9Nt1iEdtnk5DvrlaB1wSRrHiRK9JOa6GElPW/Xea9OKJDt3AQS+B/JGeRBhqbNhrmRWLJBvV7fPsRzpvJ+RwX7gO6M8W/2FtWOMo1knkyjHNoozHsFmsTY/FZcJuiKKY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1720015102; c=relaxed/simple; bh=GPo+wJphr/bQOMx/lM4LzA7A2cyA6HvGGdXEE2sIFAw=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=Uxr6K/s2RQyZW2VOljWcNYGdiw2AAaf3G7E3Sk9Bk/1ih/D2yAHm7ftFO3EwMlDvO/DZH2047vArv2KltxPjjYjkJnK/KHY7DLpv5N0ATzZKqhBcS8ZFprznX6umt8B2DNtnqfVxScYM26Qvlef+RKtrunphVDjiXyVzBPSv4Gk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=C0OUbqR6; arc=none smtp.client-ip=209.85.214.169 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="C0OUbqR6" Received: by mail-pl1-f169.google.com with SMTP id d9443c01a7336-1f9de13d6baso35316585ad.2 for ; Wed, 03 Jul 2024 06:58:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1720015100; x=1720619900; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=4RAjRwOw8tT55AuT39YuPxyEG/zw/RdiUBjfj6NkTC4=; b=C0OUbqR6d4H4tA44iWdWF50CKfYNoDUcK6jGQyfhvbqsw8HhkroTYwd2hqxmP82pkd 2B/30D+lqwDPajwaK4tVSqlSgDwFgiO1HuMMm6Q/mxv7+4rNnYSl72LVNxSImQCsRGbg M4ss9cK8czv5BV/4lm824VvS7hfTJI6qK03ObORpG+WJ5l/n8yjEocuZXtqMrd4oyrdh B9AC2DOXAkdWvBMqDyoB9LCMYg1KBHN2f7brt5H3XaxhWFc64OoVfwShZZjGeH013vgc GH3QWxFl7tgAY7+vU7yuv6oHq7GQ08lL4kaF86vE84UE4I29cNaFxXyHC/2qOc9XcFUb D0xg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1720015100; x=1720619900; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=4RAjRwOw8tT55AuT39YuPxyEG/zw/RdiUBjfj6NkTC4=; b=ftjAvL95x4HtmOFR0kLwRnxOI5yonhPPVC/SzvsIqsFWTdH78f61PdlWNJC1AzmqlA +dwLtnH/2+CwEkVtKT0ASmUS2AZu6bBZqVQv+OycZ9nskU3qN1HhjGlJEtCioYLLqc7f gpj+a46cFh60Bfx9V67RlDYGrIxCd9/9SL44T71GX3qweFdinEltfSxJwureB6bVPUZX OR6RgQ7rXnjKa4+mE4U64bHWM5osCY0n5WLOR0UTtzylti8QKMBGmQYrcrXQwIgzhsDQ C83Y2kGTGq97j+UWpR28tPNLFL4opBDYUaJnjeD8+NK+49vbVg4uYD951SJJQQyJLclz 2MkA== X-Gm-Message-State: AOJu0YwnagvXExx083A2pK7uik4YcYQ+KHx/LZCv/lj5cQUtq7T7fPuf csOwzCth77aZID00EvFAuXxZkXPoiOZJEZSqzfRlCGS1eBdX8Z0qmJCyAkh9 X-Google-Smtp-Source: AGHT+IGqyij6FvxnYBjKxrDiCcH/EAa9bJaJsB80YBvdo4gCwyoRyndvUn9lXdnxZjaHaURrR+AoGA== X-Received: by 2002:a17:90b:33c4:b0:2c9:649c:5e07 with SMTP id 98e67ed59e1d1-2c9649c6a5dmr3138227a91.8.1720015099779; Wed, 03 Jul 2024 06:58:19 -0700 (PDT) Received: from localhost ([2605:52c0:1:4cf:6c5a:92ff:fe25:ceff]) by smtp.gmail.com with ESMTPSA id 98e67ed59e1d1-2c91d3bc540sm10748751a91.44.2024.07.03.06.58.19 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 03 Jul 2024 06:58:19 -0700 (PDT) Date: Wed, 3 Jul 2024 21:58:19 +0800 From: shejialuo To: git@vger.kernel.org Cc: Patrick Steinhardt , Karthik Nayak , Junio C Hamano , Eric Sunshine Subject: [GSoC][PATCH v7 6/9] builtin/fsck: add `git-refs verify` child process Message-ID: References: Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: Introduce a new function "fsck_refs" that initializes and runs a child process to execute the "git-refs verify" command. Mentored-by: Patrick Steinhardt Mentored-by: Karthik Nayak Signed-off-by: shejialuo --- builtin/fsck.c | 17 +++++++++++++++++ 1 file changed, 17 insertions(+) diff --git a/builtin/fsck.c b/builtin/fsck.c index de34538c4f..ec3357722c 100644 --- a/builtin/fsck.c +++ b/builtin/fsck.c @@ -897,6 +897,21 @@ static int check_pack_rev_indexes(struct repository *r, int show_progress) return res; } +static void fsck_refs(void) +{ + struct child_process refs_verify = CHILD_PROCESS_INIT; + child_process_init(&refs_verify); + refs_verify.git_cmd = 1; + strvec_pushl(&refs_verify.args, "refs", "verify", NULL); + if (verbose) + strvec_push(&refs_verify.args, "--verbose"); + if (check_strict) + strvec_push(&refs_verify.args, "--strict"); + + if (run_command(&refs_verify)) + errors_found |= ERROR_REFS; +} + static char const * const fsck_usage[] = { N_("git fsck [--tags] [--root] [--unreachable] [--cache] [--no-reflogs]\n" " [--[no-]full] [--strict] [--verbose] [--lost-found]\n" @@ -1066,6 +1081,8 @@ int cmd_fsck(int argc, const char **argv, const char *prefix) check_connectivity(); + fsck_refs(); + if (the_repository->settings.core_commit_graph) { struct child_process commit_graph_verify = CHILD_PROCESS_INIT; From patchwork Wed Jul 3 13:58:47 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: shejialuo X-Patchwork-Id: 13722270 Received: from mail-pj1-f50.google.com (mail-pj1-f50.google.com [209.85.216.50]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E556F1E869 for ; Wed, 3 Jul 2024 13:58:49 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.50 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1720015131; cv=none; b=r+bLGbiifVkEBExnOTNdJQaTJVULEFkCmH5AfLHWcrN7zczvr99Jj2GrUHSZYkKEpEzfgmdgO4/WxCSyI3olJCLpPJofZ5OWcssbxVYWEt14AWAPVm27E6C6S+wXuIpKPFXOxzlGCCF99LBjULIRzOj+RfcBTCVK2rE1+Cw678A= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1720015131; c=relaxed/simple; bh=S0bP3/AX3MX3DxFnCOPiXKJrLhQV20Me5gL9Jqq5Gbo=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=Qa/9vPzim3hnwfoW+k5DCycoEySHw+6Nwt0GTfBqXDhOnL5xWjXe+CIjHkzTT9H51RO1AxaRdL4Y6tGInH8FawzHQzJCg4z8/lp9xuZ2ue7eQoVqslAxzI4bk9+o0fzpRZ0q4IkRZZrLmvd8glpPdOmQSDp/e7KRRNy///cbmeY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=nW5ewd4o; arc=none smtp.client-ip=209.85.216.50 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="nW5ewd4o" Received: by mail-pj1-f50.google.com with SMTP id 98e67ed59e1d1-2c98b22638bso118807a91.1 for ; Wed, 03 Jul 2024 06:58:49 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1720015129; x=1720619929; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=57TGoVIpBiCRGE1WDkAeGjYn5Dq+phJkTaZrEfb0Ta0=; b=nW5ewd4ocJjkpT3KBdYCv0DLx6Um6gSEPBU3n4c40GLCJSTnwSgB66bxWUz7GE6rVq k0UmRP1la0lmOWNqMxj95l5pcs2bC2a9m/TCv4F2Nc6kZSBckJDcEGAnA7Ds5S9b4oAV 0Mvrc3GlANdsNJ+SPx+sDfa1MnOqwJjxs32D7Ep3ApA8IflBluUTShGaGfucZHWCfRgI 9zfORqeUmZs4iYZmr77ebf7TvyO0WTmQsnB+22LAEeJ0FdziYguLpJyAxNzhzsTObHVo pAQQzDfpxlnuLcv+Z+HFX0n6OzacKjh8UhWCdw0vGHHPP0XQf3tWVRO8TBJ+nz900Od8 0rdA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1720015129; x=1720619929; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=57TGoVIpBiCRGE1WDkAeGjYn5Dq+phJkTaZrEfb0Ta0=; b=D2taEQA5tbkNkvWiACJ0rWF8cb/QWNFaN6wcsVMEL8YBNLM/b8vPi5daR2RA8QP9aO vC/2VTM4TuvG4oC3lk88JUfh6V0SAgNKYUGZ4PtdOp8jkF9sAPb5HBFxWJL1eL/tn659 hRTs1Ri5BKE+nWEvuOTfTvJCl1YR8Z98Eu2l3PThUz1fn3PZQ4G2bBwmrIl392XRAHz8 otjyWzjmlPm6FpAeN4bDHfQZxWPfkA89IKVjJr84xYjfeCo1g5pS2ObutjP42a3U/dF/ x3KUD+85UNbIiqOsb7zV7AUDavFwux62xfFfkALYz0NKWsJJ6i5rajLWbdxv/YJe0/WM dOlg== X-Gm-Message-State: AOJu0YyxcOs1ESwtPk6iXBBcU/aVZUxkV037VqmSZil3YWbulwm72bl6 L1jPr6qOyFwRhoiMif44m1FFbYhibjfy2SQ3WsVuggvXvPHXnfs3xsV5wU6C X-Google-Smtp-Source: AGHT+IGXNrGno+4w0I8P0EYqiymJKUIu2yOz9CAgtS3c8H+JAdovBtIv3Nfs3m8PT5frPCq41roE3Q== X-Received: by 2002:a17:90a:e291:b0:2c9:648f:f0ef with SMTP id 98e67ed59e1d1-2c97acf5810mr2919588a91.9.1720015128629; Wed, 03 Jul 2024 06:58:48 -0700 (PDT) Received: from localhost ([2605:52c0:1:4cf:6c5a:92ff:fe25:ceff]) by smtp.gmail.com with ESMTPSA id 98e67ed59e1d1-2c91d3e7fe9sm10768155a91.50.2024.07.03.06.58.46 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 03 Jul 2024 06:58:48 -0700 (PDT) Date: Wed, 3 Jul 2024 21:58:47 +0800 From: shejialuo To: git@vger.kernel.org Cc: Patrick Steinhardt , Karthik Nayak , Junio C Hamano , Eric Sunshine Subject: [GSoC][PATCH v7 7/9] files-backend: add unified interface for refs scanning Message-ID: References: Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: For refs and reflogs, we need to scan its corresponding directories to check every regular file or symbolic link which shares the same pattern. Introduce a unified interface for scanning directories for files-backend. Mentored-by: Patrick Steinhardt Mentored-by: Karthik Nayak Signed-off-by: shejialuo --- refs/files-backend.c | 77 +++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 76 insertions(+), 1 deletion(-) diff --git a/refs/files-backend.c b/refs/files-backend.c index d89eeda8ef..84acb58782 100644 --- a/refs/files-backend.c +++ b/refs/files-backend.c @@ -6,6 +6,7 @@ #include "../gettext.h" #include "../hash.h" #include "../hex.h" +#include "../fsck.h" #include "../refs.h" #include "refs-internal.h" #include "ref-cache.h" @@ -3408,6 +3409,78 @@ static int files_ref_store_remove_on_disk(struct ref_store *ref_store, return ret; } +/* + * For refs and reflogs, they share a unified interface when scanning + * the whole directory. This function is used as the callback for each + * regular file or symlink in the directory. + */ +typedef int (*files_fsck_refs_fn)(struct fsck_options *o, + const char *gitdir, + const char *refs_check_dir, + struct dir_iterator *iter); + +static int files_fsck_refs_dir(struct ref_store *ref_store, + struct fsck_options *o, + const char *refs_check_dir, + files_fsck_refs_fn *fsck_refs_fns) +{ + const char *gitdir = ref_store->gitdir; + struct strbuf sb = STRBUF_INIT; + struct dir_iterator *iter; + int iter_status; + int ret = 0; + + strbuf_addf(&sb, "%s/%s", gitdir, refs_check_dir); + + iter = dir_iterator_begin(sb.buf, 0); + + if (!iter) { + ret = error_errno("cannot open directory %s", sb.buf); + goto out; + } + + while ((iter_status = dir_iterator_advance(iter)) == ITER_OK) { + if (S_ISDIR(iter->st.st_mode)) { + continue; + } else if (S_ISREG(iter->st.st_mode) || + S_ISLNK(iter->st.st_mode)) { + if (o->verbose_refs) + fprintf_ln(stderr, "Checking %s/%s", + refs_check_dir, iter->relative_path); + for (size_t i = 0; fsck_refs_fns[i]; i++) { + if (fsck_refs_fns[i](o, gitdir, refs_check_dir, iter)) + ret = -1; + } + } else { + ret = error(_("unexpected file type for '%s'"), + iter->basename); + } + } + + if (iter_status != ITER_DONE) + ret = error(_("failed to iterate over '%s'"), sb.buf); + +out: + strbuf_release(&sb); + return ret; +} + +static int files_fsck_refs(struct ref_store *ref_store, + struct fsck_options *o) +{ + int ret; + files_fsck_refs_fn fsck_refs_fns[]= { + NULL + }; + + if (o->verbose_refs) + fprintf_ln(stderr, "Checking references consistency"); + + ret = files_fsck_refs_dir(ref_store, o, "refs", fsck_refs_fns); + + return ret; +} + static int files_fsck(struct ref_store *ref_store, struct fsck_options *o) { @@ -3415,7 +3488,9 @@ static int files_fsck(struct ref_store *ref_store, struct files_ref_store *refs = files_downcast(ref_store, REF_STORE_READ, "fsck"); - ret = refs->packed_ref_store->be->fsck(refs->packed_ref_store, o); + ret = refs->packed_ref_store->be->fsck(refs->packed_ref_store, o) + | files_fsck_refs(ref_store, o); + return ret; } From patchwork Wed Jul 3 13:59:02 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: shejialuo X-Patchwork-Id: 13722271 Received: from mail-pl1-f170.google.com (mail-pl1-f170.google.com [209.85.214.170]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B1CFC1741CF for ; Wed, 3 Jul 2024 13:59:03 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.170 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1720015145; cv=none; b=iNfFEwrI2XPUYZ6tYuA6BiBfoKMM376gQPWBIyoQfgMiH9JA/XLQehFYe6IlURy4QUHCJAQuJqi2myOK7+aW6YVgYQzlA3F4Kzctb8uLBcB80z4pdIH6iMLVvo+qVQBfLi2b4o3sTcS7PDYj4y2Q5yHZjArm+BcCBxXVL+DOk5c= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1720015145; c=relaxed/simple; bh=rsrODM2J+BWsKhm3xax6cqZkAC7HLfeiG3WeFmJYGQ4=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=f7x2rk+QW1HVS8TjLc5iKpZTAPKyc7NpYdjLpYfTk8ieiGyEKC1ZQIStuXCy5cfUDruOo4zokOUdJ4wsZZPTsFiC3ZOGzdmMyK/4iyyPZDCPS8y7iwxu2ZaI+XeqSD72Xv4fscmbQ59diO6FNBTE8e4IDVrdHYaKd+k8SQFb2IY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=Mj5usHn4; arc=none smtp.client-ip=209.85.214.170 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="Mj5usHn4" Received: by mail-pl1-f170.google.com with SMTP id d9443c01a7336-1fb1c918860so3647925ad.1 for ; Wed, 03 Jul 2024 06:59:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1720015142; x=1720619942; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=YtZFlVnAe4qE/y0j2Rt0N7Q1NZXS8OgfPHqwdyraSzA=; b=Mj5usHn41u+t+2Q0S4aWrnvLqI+dTw1WfVkFYEQWjV5yMl6GrYhSsqcKUEPG48RoBP N8e/OaJaCNF72VieT59Cinme9DbS+AUZLKbpZt2TgCnzGil9JONqs9SNL6/PJAuUWydG lqBQ+MRtCdOgiWFXGA9zsSU+lu3UeGBO/z4yMNxk/hM3ur9sF7c48HYNBDIoa/TqoSIu 7DvIbMze78bXhp8RICyZmN5EC381GjSWel4N5ZnHKUp3pV36G1TcPrTapv33v38Ep7hG rPA/f864WSxYE9naXDfpiIqvzeLUDs7jmItFH1RvSix4zEPPl6nzMnyfmRq4opGG07HI kWmw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1720015142; x=1720619942; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=YtZFlVnAe4qE/y0j2Rt0N7Q1NZXS8OgfPHqwdyraSzA=; b=Fq+x+rXs5Euou6on+lrsw+/YUwjbl1tR42ZW2sKDr6wVk1UB5/1o9XeuqZPeZSzlnO lmnaexV2IcBnYxdQE9Zh1Iad1cRAo8QrVVkI/7oldlK0uh+TFSkSFpT4S2XXfzu5KzI0 Wwq40/nUdPsOM6nuDXbkPbHDrQsD9QNPahNnD943IM/QELR75wbhEIiYUGtyw9KHr6R3 DgZXCIPEO4EQDeSJuDPLm4pyxEeyz19iLyIyoa32qxjIr1yrndeIIDkJR+TxBVPLHMuN zYx3MiqJa1ERPhsZlBlG3JWe0AOdUVyJNYD8blaOlw/1ZgKYYH/hXMQuEsWZxbbUYTq1 4b1w== X-Gm-Message-State: AOJu0YyvS+NLMBeUHKRwWJZfn8r9rtxF/qxzFRoRETI3zkJmXVyUMegS K/95H7FR9I+IGyrWp4Ps9Hi40MUg+R1DGjQse8BHAvpzx4g3FHdHEYYElEgg X-Google-Smtp-Source: AGHT+IFOXTkvcrhsdfCH1AjM22KxQ4+1M+2YqZ9keYJdFF3dOunzvk0Zflfnv1BKVka2XLk8eVPZiA== X-Received: by 2002:a17:903:1245:b0:1f8:44f8:a357 with SMTP id d9443c01a7336-1fb1a06ed94mr29366855ad.17.1720015142385; Wed, 03 Jul 2024 06:59:02 -0700 (PDT) Received: from localhost ([2605:52c0:1:4cf:6c5a:92ff:fe25:ceff]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-1fac157b3b7sm103067855ad.242.2024.07.03.06.59.01 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 03 Jul 2024 06:59:01 -0700 (PDT) Date: Wed, 3 Jul 2024 21:59:02 +0800 From: shejialuo To: git@vger.kernel.org Cc: Patrick Steinhardt , Karthik Nayak , Junio C Hamano , Eric Sunshine Subject: [GSoC][PATCH v7 8/9] fsck: add ref name check for files backend Message-ID: References: Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: The git-fsck(1) only implicitly checks the reference, it does not fully check refs with bad format name such as standalone "@" and name ending with ".lock". In order to provide such checks, add a new fsck message id "badRefName" with default ERROR type. Use existing "check_refname_format" to explicit check the ref name. And add a new unit test to verify the functionality. Mentored-by: Patrick Steinhardt Mentored-by: Karthik Nayak Signed-off-by: shejialuo --- Documentation/fsck-msgids.txt | 3 + fsck.h | 1 + refs/files-backend.c | 20 +++++++ t/t0602-reffiles-fsck.sh | 101 ++++++++++++++++++++++++++++++++++ 4 files changed, 125 insertions(+) create mode 100755 t/t0602-reffiles-fsck.sh diff --git a/Documentation/fsck-msgids.txt b/Documentation/fsck-msgids.txt index f643585a34..dab4012246 100644 --- a/Documentation/fsck-msgids.txt +++ b/Documentation/fsck-msgids.txt @@ -19,6 +19,9 @@ `badParentSha1`:: (ERROR) A commit object has a bad parent sha1. +`badRefName`:: + (ERROR) A ref has a bad name. + `badTagName`:: (INFO) A tag has an invalid format. diff --git a/fsck.h b/fsck.h index 246055c0f9..90457d1a1f 100644 --- a/fsck.h +++ b/fsck.h @@ -31,6 +31,7 @@ enum fsck_msg_type { FUNC(BAD_NAME, ERROR) \ FUNC(BAD_OBJECT_SHA1, ERROR) \ FUNC(BAD_PARENT_SHA1, ERROR) \ + FUNC(BAD_REF_NAME, ERROR) \ FUNC(BAD_TIMEZONE, ERROR) \ FUNC(BAD_TREE, ERROR) \ FUNC(BAD_TREE_SHA1, ERROR) \ diff --git a/refs/files-backend.c b/refs/files-backend.c index 84acb58782..69a76048d3 100644 --- a/refs/files-backend.c +++ b/refs/files-backend.c @@ -3419,6 +3419,25 @@ typedef int (*files_fsck_refs_fn)(struct fsck_options *o, const char *refs_check_dir, struct dir_iterator *iter); +static int files_fsck_refs_name(struct fsck_options *o, + const char *gitdir UNUSED, + const char *refs_check_dir, + struct dir_iterator *iter) +{ + struct strbuf sb = STRBUF_INIT; + int ret = 0; + + if (check_refname_format(iter->basename, REFNAME_ALLOW_ONELEVEL)) { + strbuf_addf(&sb, "%s/%s", refs_check_dir, iter->relative_path); + ret = fsck_refs_report(o, NULL, sb.buf, + FSCK_MSG_BAD_REF_NAME, + "invalid refname format"); + } + + strbuf_release(&sb); + return ret; +} + static int files_fsck_refs_dir(struct ref_store *ref_store, struct fsck_options *o, const char *refs_check_dir, @@ -3470,6 +3489,7 @@ static int files_fsck_refs(struct ref_store *ref_store, { int ret; files_fsck_refs_fn fsck_refs_fns[]= { + files_fsck_refs_name, NULL }; diff --git a/t/t0602-reffiles-fsck.sh b/t/t0602-reffiles-fsck.sh new file mode 100755 index 0000000000..b2db58d2c6 --- /dev/null +++ b/t/t0602-reffiles-fsck.sh @@ -0,0 +1,101 @@ +#!/bin/sh + +test_description='Test reffiles backend consistency check' + +GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME=main +export GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME +GIT_TEST_DEFAULT_REF_FORMAT=files +export GIT_TEST_DEFAULT_REF_FORMAT + +. ./test-lib.sh + +test_expect_success 'ref name should be checked' ' + test_when_finished "rm -rf repo" && + git init repo && + branch_dir_prefix=.git/refs/heads && + tag_dir_prefix=.git/refs/tags && + ( + cd repo && + git commit --allow-empty -m initial && + git checkout -b branch-1 && + git tag tag-1 && + git commit --allow-empty -m second && + git checkout -b branch-2 && + git tag tag-2 && + git tag multi_hierarchy/tag-2 + ) && + ( + cd repo && + cp $branch_dir_prefix/branch-1 $branch_dir_prefix/.branch-1 && + test_must_fail git fsck 2>err && + cat >expect <<-EOF && + error: refs/heads/.branch-1: badRefName: invalid refname format + EOF + rm $branch_dir_prefix/.branch-1 && + test_cmp expect err + ) && + ( + cd repo && + cp $tag_dir_prefix/tag-1 $tag_dir_prefix/tag-1.lock && + test_must_fail git fsck 2>err && + cat >expect <<-EOF && + error: refs/tags/tag-1.lock: badRefName: invalid refname format + EOF + rm $tag_dir_prefix/tag-1.lock && + test_cmp expect err + ) && + ( + cd repo && + cp $branch_dir_prefix/branch-1 $branch_dir_prefix/@ && + test_must_fail git fsck 2>err && + cat >expect <<-EOF && + error: refs/heads/@: badRefName: invalid refname format + EOF + rm $branch_dir_prefix/@ && + test_cmp expect err + ) && + ( + cd repo && + cp $tag_dir_prefix/multi_hierarchy/tag-2 $tag_dir_prefix/multi_hierarchy/@ && + test_must_fail git fsck 2>err && + cat >expect <<-EOF && + error: refs/tags/multi_hierarchy/@: badRefName: invalid refname format + EOF + rm $tag_dir_prefix/multi_hierarchy/@ && + test_cmp expect err + ) +' + +test_expect_success 'ref name check should be adapted into fsck messages' ' + test_when_finished "rm -rf repo" && + git init repo && + branch_dir_prefix=.git/refs/heads && + tag_dir_prefix=.git/refs/tags && + ( + cd repo && + git commit --allow-empty -m initial && + git checkout -b branch-1 && + git tag tag-1 && + git commit --allow-empty -m second && + git checkout -b branch-2 && + git tag tag-2 + ) && + ( + cd repo && + cp $branch_dir_prefix/branch-1 $branch_dir_prefix/.branch-1 && + git -c fsck.badRefName=warn fsck 2>err && + cat >expect <<-EOF && + warning: refs/heads/.branch-1: badRefName: invalid refname format + EOF + rm $branch_dir_prefix/.branch-1 && + test_cmp expect err + ) && + ( + cd repo && + cp $branch_dir_prefix/branch-1 $branch_dir_prefix/@ && + git -c fsck.badRefName=ignore fsck 2>err && + test_must_be_empty err + ) +' + +test_done From patchwork Wed Jul 3 13:59:17 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: shejialuo X-Patchwork-Id: 13722272 Received: from mail-pl1-f174.google.com (mail-pl1-f174.google.com [209.85.214.174]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 2AA721741CF for ; Wed, 3 Jul 2024 13:59:19 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.174 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1720015160; cv=none; b=JLVRxhKZMaFOkZoO26zPxkdrF+0RbKiQUaMwN4q0Q5CPCue1V+TPgI+vSjqL096iakq9Spv/4sd+2RcITfi0Wkh7NNDFfYRJsxF7NLMejCCOmb5NPq8jIN0cr6nETBkIF91p1CmFhfPyrSWcV9DUqFdVZCjGqsrKZ9og5gB76C0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1720015160; c=relaxed/simple; bh=0LQLtXU1B78QCv8dBUyImGLhsgDqkwwSbL0XSVYc9PM=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=SVPzE+iqGKDDQ2AX/qNujWF+jlBgqK3jqhi+zjtSZyOuh7MG7/bMXRk0d7jN6+r2UeWHelja/dCprIl32IWPszRiEV3G0M+ZcupHwRQXM8/Y+VHxd2uS90ATT5J7uKN0gf7TzRp1ZL86l5QYSCTed0KIVdx2Tq/E3vNn0/gd/Hs= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=Ke7XPhAH; arc=none smtp.client-ip=209.85.214.174 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="Ke7XPhAH" Received: by mail-pl1-f174.google.com with SMTP id d9443c01a7336-1fa9f540f45so31528725ad.1 for ; Wed, 03 Jul 2024 06:59:19 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1720015158; x=1720619958; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=HIpbU0AiSI7yFv6vUKnqZB2UhQlcLbis3BsFeoUROSk=; b=Ke7XPhAHRvIfp4TyundqX9e2pm6EqRjyoVeRB3kPW52cUcSSovi0QVopKGf5HSD4F2 adSd8UMose4W+68655Xkga8JwzkDRnkhP1sB2ltBc81acc1xTz9mrgIH0Z3jZTjIyRkK X+9DRV2gcVjGUTREXj1oJLGiJuEA0uFFSXMqen2g2NZQNQXQuyOJvS/xc3aE7LhX0i4o Og2jhI+KIxb8MPYBc+wG9M1YDlxQfBQYMqmk/cWoi0zRy4cY1zJGWqSCZa7Rcdf/jtr0 wozAkQgFL1jrGGKKuyzeFbUhqtRMx2KNjWZzipdTUz3w2gQqwJOgWAA+5m77nPisEA8k Lz7Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1720015158; x=1720619958; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=HIpbU0AiSI7yFv6vUKnqZB2UhQlcLbis3BsFeoUROSk=; b=w2tJFbSBdgRF4d+/yTGMaqaKuhd2FK2opwdTtQABUzZBKytsMp+F4y3wUEsZApQqMv 7Ua0Ye6Q323Fi+ThDVVzXr3G8XuaLnwuYK1n6w6iEHHw6QHlVW/7no3AXC4d4P3ySMMI kd3JPqEdVdIfyLCDkULD75sxaGc/rXD6NOpDABatSMuQ8NceCBNo03q6ozFilYbFomnF 5UeQjoDqIooo51Uhmx+I5ANsTcw3mMK6u5tMiRNAXj5PwGgAMC2vtQp/MQZb1N4xD8gf ZCXAZiVE3RlYqs2JKq15l7QhfsRX5qDF4RoZq6q7jevskU6Wkr5Cr5qD3oMysr5kctVG zxoQ== X-Gm-Message-State: AOJu0YwXfcozZqSSvMPPxdvO7nwjyzTCJbACFokwgQN931YhgQ5yv1Qt /iunfhuZP1Hy9sBXr773ynEXrH2V6NrhpV8jhbzoLDETlNWOARnYUTq8QLgO X-Google-Smtp-Source: AGHT+IF3KkTI5NMvbVCCFeD92jgYH3Hsj4C1d4k/sMS4Ork4BRzPuuYTZXvyOnQfrDpI/gB5KGYISQ== X-Received: by 2002:a17:902:c951:b0:1fb:29f3:c7d4 with SMTP id d9443c01a7336-1fb29f3caacmr2077835ad.51.1720015157923; Wed, 03 Jul 2024 06:59:17 -0700 (PDT) Received: from localhost ([2605:52c0:1:4cf:6c5a:92ff:fe25:ceff]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-1fb1a075616sm13964895ad.15.2024.07.03.06.59.16 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 03 Jul 2024 06:59:17 -0700 (PDT) Date: Wed, 3 Jul 2024 21:59:17 +0800 From: shejialuo To: git@vger.kernel.org Cc: Patrick Steinhardt , Karthik Nayak , Junio C Hamano , Eric Sunshine Subject: [GSoC][PATCH v7 9/9] fsck: add ref content check for files backend Message-ID: References: Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: Enhance the git-fsck(1) command by adding a check for reference content in the files backend. The new functionality ensures that symrefs, real symbolic link and regular refs are validated correctly. In order to check the trailing content of the regular refs, add a new parameter `trailing` to `parse_loose_ref_contents`. For symrefs, `parse_loose_ref_contents` will set the "referent". However, symbolic link could be either absolute or relative. Use "strbuf_add_real_path" to read the symbolic link and convert the relative path to absolute path. Then use "skip_prefix" to make it align with symref "referent". Thus, the symrefs and symbolic links could share the same interface. Add a new function "files_fsck_symref_target" which aims at checking the following things: 1. whether the pointee is under the `refs/` directory. 2. whether the pointee name is correct. 3. whether the pointee path is a wrong type in filesystem. Last, add the following FSCK MESSAGEs: 1. "badRefContent(ERROR)": A ref has a bad content 2. "badSymrefPointee(ERROR)": The pointee of a symref is bad. 3. "trailingRefContent(WARN)": A ref content has trailing contents. Mentored-by: Patrick Steinhardt Mentored-by: Karthik Nayak Signed-off-by: shejialuo --- Documentation/fsck-msgids.txt | 9 +++ fsck.h | 3 + refs.c | 2 +- refs/files-backend.c | 145 +++++++++++++++++++++++++++++++++- refs/refs-internal.h | 5 +- t/t0602-reffiles-fsck.sh | 110 ++++++++++++++++++++++++++ 6 files changed, 269 insertions(+), 5 deletions(-) diff --git a/Documentation/fsck-msgids.txt b/Documentation/fsck-msgids.txt index dab4012246..b1630a478b 100644 --- a/Documentation/fsck-msgids.txt +++ b/Documentation/fsck-msgids.txt @@ -19,9 +19,15 @@ `badParentSha1`:: (ERROR) A commit object has a bad parent sha1. +`badRefContent`:: + (ERROR) A ref has a bad content. + `badRefName`:: (ERROR) A ref has a bad name. +`badSymrefPointee`:: + (ERROR) The pointee of a symref is bad. + `badTagName`:: (INFO) A tag has an invalid format. @@ -167,6 +173,9 @@ `nullSha1`:: (WARN) Tree contains entries pointing to a null sha1. +`trailingRefContent`:: + (WARN) A ref content has trailing contents. + `treeNotSorted`:: (ERROR) A tree is not properly sorted. diff --git a/fsck.h b/fsck.h index 90457d1a1f..637f596930 100644 --- a/fsck.h +++ b/fsck.h @@ -32,6 +32,8 @@ enum fsck_msg_type { FUNC(BAD_OBJECT_SHA1, ERROR) \ FUNC(BAD_PARENT_SHA1, ERROR) \ FUNC(BAD_REF_NAME, ERROR) \ + FUNC(BAD_REF_CONTENT, ERROR) \ + FUNC(BAD_SYMREF_POINTEE, ERROR) \ FUNC(BAD_TIMEZONE, ERROR) \ FUNC(BAD_TREE, ERROR) \ FUNC(BAD_TREE_SHA1, ERROR) \ @@ -72,6 +74,7 @@ enum fsck_msg_type { FUNC(HAS_DOTDOT, WARN) \ FUNC(HAS_DOTGIT, WARN) \ FUNC(NULL_SHA1, WARN) \ + FUNC(TRAILING_REF_CONTENT, WARN) \ FUNC(ZERO_PADDED_FILEMODE, WARN) \ FUNC(NUL_IN_COMMIT, WARN) \ FUNC(LARGE_PATHNAME, WARN) \ diff --git a/refs.c b/refs.c index 410919246b..eb82fb7d4e 100644 --- a/refs.c +++ b/refs.c @@ -1760,7 +1760,7 @@ static int refs_read_special_head(struct ref_store *ref_store, } result = parse_loose_ref_contents(content.buf, oid, referent, type, - failure_errno); + failure_errno, NULL); done: strbuf_release(&full_path); diff --git a/refs/files-backend.c b/refs/files-backend.c index 69a76048d3..d98ef45403 100644 --- a/refs/files-backend.c +++ b/refs/files-backend.c @@ -1,6 +1,7 @@ #define USE_THE_REPOSITORY_VARIABLE #include "../git-compat-util.h" +#include "../abspath.h" #include "../copy.h" #include "../environment.h" #include "../gettext.h" @@ -553,7 +554,7 @@ static int read_ref_internal(struct ref_store *ref_store, const char *refname, strbuf_rtrim(&sb_contents); buf = sb_contents.buf; - ret = parse_loose_ref_contents(buf, oid, referent, type, &myerr); + ret = parse_loose_ref_contents(buf, oid, referent, type, &myerr, NULL); out: if (ret && !myerr) @@ -589,7 +590,7 @@ static int files_read_symbolic_ref(struct ref_store *ref_store, const char *refn int parse_loose_ref_contents(const char *buf, struct object_id *oid, struct strbuf *referent, unsigned int *type, - int *failure_errno) + int *failure_errno, const char **trailing) { const char *p; if (skip_prefix(buf, "ref:", &buf)) { @@ -611,6 +612,10 @@ int parse_loose_ref_contents(const char *buf, struct object_id *oid, *failure_errno = EINVAL; return -1; } + + if (trailing) + *trailing = p; + return 0; } @@ -3438,6 +3443,141 @@ static int files_fsck_refs_name(struct fsck_options *o, return ret; } +/* + * Check the symref "pointee_name" and "pointee_path". The caller should + * make sure that "pointee_path" is absolute. For symbolic ref, "pointee_name" + * would be the content after "refs:". For symblic link, "pointee_name" would + * be the relative path agaignst "gitdir". + */ +static int files_fsck_symref_target(struct fsck_options *o, + const char *refname, + const char *pointee_name, + const char *pointee_path) +{ + const char *p = NULL; + struct stat st; + int ret = 0; + + if (!skip_prefix(pointee_name, "refs/", &p)) { + + ret = fsck_refs_report(o, NULL, refname, + FSCK_MSG_BAD_SYMREF_POINTEE, + "point to target out of refs hierarchy"); + goto out; + } + + if (check_refname_format(pointee_name, 0)) { + ret = fsck_refs_report(o, NULL, refname, + FSCK_MSG_BAD_SYMREF_POINTEE, + "point to invalid refname"); + } + + if (lstat(pointee_path, &st) < 0) + goto out; + + if (!S_ISREG(st.st_mode) && !S_ISLNK(st.st_mode)) { + ret = fsck_refs_report(o, NULL, refname, + FSCK_MSG_BAD_SYMREF_POINTEE, + "point to invalid target"); + goto out; + } +out: + return ret; +} + +static int files_fsck_refs_content(struct fsck_options *o, + const char *gitdir, + const char *refs_check_dir, + struct dir_iterator *iter) +{ + struct strbuf pointee_path = STRBUF_INIT, + ref_content = STRBUF_INIT, + abs_gitdir = STRBUF_INIT, + referent = STRBUF_INIT, + refname = STRBUF_INIT; + const char *trailing = NULL; + int failure_errno = 0; + unsigned int type = 0; + struct object_id oid; + int ret = 0; + + strbuf_addf(&refname, "%s/%s", refs_check_dir, iter->relative_path); + + /* + * If the file is a symlink, we need to only check the connectivity + * of the destination object. + */ + if (S_ISLNK(iter->st.st_mode)) { + const char *pointee_name = NULL; + + strbuf_add_real_path(&pointee_path, iter->path.buf); + + strbuf_add_absolute_path(&abs_gitdir, gitdir); + strbuf_normalize_path(&abs_gitdir); + if (!is_dir_sep(abs_gitdir.buf[abs_gitdir.len - 1])) + strbuf_addch(&abs_gitdir, '/'); + + if (!skip_prefix(pointee_path.buf, + abs_gitdir.buf, &pointee_name)) { + ret = fsck_refs_report(o, NULL, refname.buf, + FSCK_MSG_BAD_SYMREF_POINTEE, + "point to target outside gitdir"); + goto clean; + } + + ret = files_fsck_symref_target(o, refname.buf, pointee_name, + pointee_path.buf); + goto clean; + } + + if (strbuf_read_file(&ref_content, iter->path.buf, 0) < 0) { + ret = error_errno(_("%s/%s: unable to read the ref"), + refs_check_dir, iter->relative_path); + goto clean; + } + + if (parse_loose_ref_contents(ref_content.buf, &oid, + &referent, &type, + &failure_errno, &trailing)) { + ret = fsck_refs_report(o, NULL, refname.buf, + FSCK_MSG_BAD_REF_CONTENT, + "invalid ref content"); + goto clean; + } + + /* + * If the ref is a symref, we need to check the destination name and + * connectivity. + */ + if (referent.len && (type & REF_ISSYMREF)) { + strbuf_addf(&pointee_path, "%s/%s", gitdir, referent.buf); + strbuf_rtrim(&referent); + + ret = files_fsck_symref_target(o, refname.buf, referent.buf, + pointee_path.buf); + goto clean; + } else { + /* + * Only regular refs could have a trailing garbage. Should + * be reported as a warning. + */ + if (trailing && (*trailing != '\0' && *trailing != '\n')) { + ret = fsck_refs_report(o, NULL, refname.buf, + FSCK_MSG_TRAILING_REF_CONTENT, + "trailing garbage in ref"); + goto clean; + } + } + +clean: + strbuf_release(&abs_gitdir); + strbuf_release(&pointee_path); + strbuf_release(&refname); + strbuf_release(&ref_content); + strbuf_release(&referent); + return ret; +} + static int files_fsck_refs_dir(struct ref_store *ref_store, struct fsck_options *o, const char *refs_check_dir, @@ -3490,6 +3630,7 @@ static int files_fsck_refs(struct ref_store *ref_store, int ret; files_fsck_refs_fn fsck_refs_fns[]= { files_fsck_refs_name, + files_fsck_refs_content, NULL }; diff --git a/refs/refs-internal.h b/refs/refs-internal.h index a905e187cd..2fabf41d14 100644 --- a/refs/refs-internal.h +++ b/refs/refs-internal.h @@ -709,11 +709,12 @@ struct ref_store { /* * Parse contents of a loose ref file. *failure_errno maybe be set to EINVAL for - * invalid contents. + * invalid contents. Also *trailing is set to the first character after the + * refname or NULL if the referent is not empty. */ int parse_loose_ref_contents(const char *buf, struct object_id *oid, struct strbuf *referent, unsigned int *type, - int *failure_errno); + int *failure_errno, const char **trailing); /* * Fill in the generic part of refs and add it to our collection of diff --git a/t/t0602-reffiles-fsck.sh b/t/t0602-reffiles-fsck.sh index b2db58d2c6..35bf40ee64 100755 --- a/t/t0602-reffiles-fsck.sh +++ b/t/t0602-reffiles-fsck.sh @@ -98,4 +98,114 @@ test_expect_success 'ref name check should be adapted into fsck messages' ' ) ' +test_expect_success 'regular ref content should be checked' ' + test_when_finished "rm -rf repo" && + git init repo && + branch_dir_prefix=.git/refs/heads && + tag_dir_prefix=.git/refs/tags && + ( + cd repo && + git commit --allow-empty -m initial && + git checkout -b branch-1 && + git tag tag-1 && + git commit --allow-empty -m second && + git checkout -b branch-2 && + git tag tag-2 && + git checkout -b a/b/tag-2 + ) && + ( + cd repo && + printf "%s garbage" "$(git rev-parse branch-1)" > $branch_dir_prefix/branch-1-garbage && + git fsck 2>err && + cat >expect <<-EOF && + warning: refs/heads/branch-1-garbage: trailingRefContent: trailing garbage in ref + EOF + rm $branch_dir_prefix/branch-1-garbage && + test_cmp expect err + ) && + ( + cd repo && + printf "%s garbage" "$(git rev-parse tag-1)" > $tag_dir_prefix/tag-1-garbage && + test_must_fail git -c fsck.trailingRefContent=error fsck 2>err && + cat >expect <<-EOF && + error: refs/tags/tag-1-garbage: trailingRefContent: trailing garbage in ref + EOF + rm $tag_dir_prefix/tag-1-garbage && + test_cmp expect err + ) && + ( + cd repo && + printf "%s " "$(git rev-parse tag-2)" > $tag_dir_prefix/tag-2-garbage && + git fsck 2>err && + cat >expect <<-EOF && + warning: refs/tags/tag-2-garbage: trailingRefContent: trailing garbage in ref + EOF + rm $tag_dir_prefix/tag-2-garbage && + test_cmp expect err + ) && + ( + cd repo && + printf "xfsazqfxcadas" > $tag_dir_prefix/tag-2-bad && + test_must_fail git refs verify 2>err && + cat >expect <<-EOF && + error: refs/tags/tag-2-bad: badRefContent: invalid ref content + EOF + rm $tag_dir_prefix/tag-2-bad && + test_cmp expect err + ) && + ( + cd repo && + printf "xfsazqfxcadas" > $branch_dir_prefix/a/b/branch-2-bad && + test_must_fail git refs verify 2>err && + cat >expect <<-EOF && + error: refs/heads/a/b/branch-2-bad: badRefContent: invalid ref content + EOF + rm $branch_dir_prefix/a/b/branch-2-bad && + test_cmp expect err + ) +' + +test_expect_success 'symbolic ref content should be checked' ' + test_when_finished "rm -rf repo" && + git init repo && + branch_dir_prefix=.git/refs/heads && + tag_dir_prefix=.git/refs/tags && + ( + cd repo && + git commit --allow-empty -m initial && + git checkout -b branch-1 && + git tag tag-1 + ) && + ( + cd repo && + printf "ref: refs/heads/.branch" > $branch_dir_prefix/branch-2-bad && + test_must_fail git refs verify 2>err && + cat >expect <<-EOF && + error: refs/heads/branch-2-bad: badSymrefPointee: point to invalid refname + EOF + rm $branch_dir_prefix/branch-2-bad && + test_cmp expect err + ) && + ( + cd repo && + printf "ref: refs/heads" > $branch_dir_prefix/branch-2-bad && + test_must_fail git refs verify 2>err && + cat >expect <<-EOF && + error: refs/heads/branch-2-bad: badSymrefPointee: point to invalid target + EOF + rm $branch_dir_prefix/branch-2-bad && + test_cmp expect err + ) && + ( + cd repo && + printf "ref: logs/maint-v2.45" > $branch_dir_prefix/branch-2-bad && + test_must_fail git refs verify 2>err && + cat >expect <<-EOF && + error: refs/heads/branch-2-bad: badSymrefPointee: point to target out of refs hierarchy + EOF + rm $branch_dir_prefix/branch-2-bad && + test_cmp expect err + ) +' + test_done