From patchwork Tue Apr 8 10:24:09 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Patrick Steinhardt X-Patchwork-Id: 14042664 Received: from fout-a6-smtp.messagingengine.com (fout-a6-smtp.messagingengine.com [103.168.172.149]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 333FA268FCB for ; Tue, 8 Apr 2025 10:24:16 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=103.168.172.149 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1744107860; cv=none; b=aIzCQGqhZhNgu1IS3o7n9HjBujlgCbSpyG+T92K5cWrUCOe3LZTbKaq300rvx7fiQXTpLjN/Pxni3VoXSFILF6oObs3EzQEprhbWMPM8KjZWzSRnYBksaCzTgvIkuo9QTz1RkYUy8LpFiFwjd82297e25cUEEU5rGZ8ojLeE4JY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1744107860; c=relaxed/simple; bh=9QSEZWNmpalTG5h0R1RyIUqGKDFcyXogNOi8gPmlsoA=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=M5h2zecNmMkI9KPSoyli2tNmnO13eQPbuddR8EqPRbHuPL/Pd1LCix3VXfgYt7/Rp91RyM6tuMAjPDZiuTJDNENoOexSDKSMJslY8WWMWbMc0ckcOjc9qCep4+nb9vzn7DYpT4l/0D2QxOMJmC4NMakXkePV4sXtKKhPuLJAR2I= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=pks.im; spf=pass smtp.mailfrom=pks.im; dkim=pass (2048-bit key) header.d=pks.im header.i=@pks.im header.b=rJuTHzoi; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b=bRzQugGK; arc=none smtp.client-ip=103.168.172.149 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=pks.im Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=pks.im Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=pks.im header.i=@pks.im header.b="rJuTHzoi"; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b="bRzQugGK" Received: from phl-compute-02.internal (phl-compute-02.phl.internal [10.202.2.42]) by mailfout.phl.internal (Postfix) with ESMTP id 33D1D1380140 for ; Tue, 8 Apr 2025 06:24:16 -0400 (EDT) Received: from phl-mailfrontend-01 ([10.202.2.162]) by phl-compute-02.internal (MEProxy); Tue, 08 Apr 2025 06:24:16 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=pks.im; h=cc:cc :content-transfer-encoding:content-type:content-type:date:date :from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:subject:subject:to:to; s=fm2; t=1744107856; x=1744194256; bh=LuszaVirLVoOqFSELgrKLswbiermHa0uQlpCascoWug=; b= rJuTHzoie1ChIlb4KEdRIcddVHVXg0qpBZrezGdcgxcPVJJN2NYBSKT/2jm7xCtY U+s6ibHvcqM4c9ysGZkDYjWdI+PW1yY33sg6Q9R2DYBJa+IpTcyW7NnkhnmRHN6u oDakI3XoRbXOjOmu4Zhpjh1p1gZKWqlIIUapVKIFSXnI8V+jUDD6xrEC4lvlGjD8 wvNgVLG8ee+P++JGJ62LSDR3YXZ81LT1rO7sXi5Hj70ckS2mnLmwR58NVHXvZBja L/NNgBt1FsUvj/uf50xGNq9jWnbTjDAglHDAdGAkyoXAD+Rv8eF9Mx9fnILOTjd1 PWNSjJvXnO1WMt4kDDO/2A== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-transfer-encoding :content-type:content-type:date:date:feedback-id:feedback-id :from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:subject:subject:to:to:x-me-proxy :x-me-sender:x-me-sender:x-sasl-enc; s=fm2; t=1744107856; x= 1744194256; bh=LuszaVirLVoOqFSELgrKLswbiermHa0uQlpCascoWug=; b=b RzQugGKL/d46XzzkJ4UeqAgWGCEI4ItviwFxt9FnlCEdn0wyHhvdufw35Jfd4tQV gxvz8z2GUwPPtpjKZ4gvgT9ntJH6FTQhbbtzlBItOEemMv0FVsIm8kNOVQ2p+Dq2 i4Yq0HsmdYm6ofvFZdWWoJZRWxR7jpoM6JnzjTF6hH7a17yeDcKf+Ia3EYZLnvG/ kVkz/XiByCAw5q7Xy+ITbKQCLNgFfbw+6dK7r5d2OJMe/YJ+iUfKZwFq06VzJ2h2 Ce/U75EYoR4r+GANEmrUAG1uhY5kFWmuvFWGjJwUhjsORQDileLFxFi9m8+R01Y5 vWYqS9E1/aTrOGKbaVVLw== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeefvddrtddtgddvtddvkeegucetufdoteggodetrf dotffvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdggtfgfnhhsuhgsshgtrhhisggv pdfurfetoffkrfgpnffqhgenuceurghilhhouhhtmecufedttdenucenucfjughrpefhff fugggtgffkfhgjvfevofesthejredtredtjeenucfhrhhomheprfgrthhrihgtkhcuufht vghinhhhrghrughtuceophhssehpkhhsrdhimheqnecuggftrfgrthhtvghrnhepffeuie dujedvkeehuedvkeefffeivdeuleetkeduheejteekgedvudfgtdfgieelnecuvehluhhs thgvrhfuihiivgeptdenucfrrghrrghmpehmrghilhhfrhhomhepphhssehpkhhsrdhimh dpnhgspghrtghpthhtohepuddpmhhouggvpehsmhhtphhouhhtpdhrtghpthhtohepghhi thesvhhgvghrrdhkvghrnhgvlhdrohhrgh X-ME-Proxy: Feedback-ID: i197146af:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA for ; Tue, 8 Apr 2025 06:24:15 -0400 (EDT) Received: by vm-mail (OpenSMTPD) with ESMTPSA id 92251c74 (TLSv1.3:TLS_AES_256_GCM_SHA384:256:NO) for ; Tue, 8 Apr 2025 10:24:13 +0000 (UTC) From: Patrick Steinhardt Date: Tue, 08 Apr 2025 12:24:09 +0200 Subject: [PATCH 1/9] object-file: move `safe_create_leading_directories()` into "dir.c" Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20250408-pks-split-object-file-v1-1-f1fd50191143@pks.im> References: <20250408-pks-split-object-file-v1-0-f1fd50191143@pks.im> In-Reply-To: <20250408-pks-split-object-file-v1-0-f1fd50191143@pks.im> To: git@vger.kernel.org Cc: X-Mailer: b4 0.14.2 The `safe_create_leading_directories()` function and its relatives are located in "object-file.c", which is not a good fit as they provide generic functionality not related to objects at all. Move them into "dir.c". Signed-off-by: Patrick Steinhardt --- builtin/bugreport.c | 2 +- builtin/credential-cache--daemon.c | 2 +- builtin/diagnose.c | 2 +- builtin/fsck.c | 1 + builtin/gc.c | 2 +- builtin/init-db.c | 2 +- builtin/log.c | 2 +- commit-graph.c | 1 + dir.c | 107 ++++++++++++++++++++++++++++++++++++- dir.h | 35 ++++++++++++ midx-write.c | 1 + object-file.c | 106 ------------------------------------ object-file.h | 35 ------------ 13 files changed, 150 insertions(+), 148 deletions(-) diff --git a/builtin/bugreport.c b/builtin/bugreport.c index 66d64bfd5ae..d07fa91c247 100644 --- a/builtin/bugreport.c +++ b/builtin/bugreport.c @@ -1,6 +1,7 @@ #define USE_THE_REPOSITORY_VARIABLE #include "builtin.h" #include "abspath.h" +#include "dir.h" #include "editor.h" #include "gettext.h" #include "parse-options.h" @@ -10,7 +11,6 @@ #include "hook.h" #include "hook-list.h" #include "diagnose.h" -#include "object-file.h" #include "setup.h" #include "version.h" diff --git a/builtin/credential-cache--daemon.c b/builtin/credential-cache--daemon.c index e707618e743..80d29b4f5c0 100644 --- a/builtin/credential-cache--daemon.c +++ b/builtin/credential-cache--daemon.c @@ -1,8 +1,8 @@ #define USE_THE_REPOSITORY_VARIABLE #include "builtin.h" #include "abspath.h" +#include "dir.h" #include "gettext.h" -#include "object-file.h" #include "parse-options.h" #ifndef NO_UNIX_SOCKETS diff --git a/builtin/diagnose.c b/builtin/diagnose.c index 33c39bd5981..d5dadd6a48b 100644 --- a/builtin/diagnose.c +++ b/builtin/diagnose.c @@ -2,8 +2,8 @@ #include "builtin.h" #include "abspath.h" +#include "dir.h" #include "gettext.h" -#include "object-file.h" #include "parse-options.h" #include "diagnose.h" diff --git a/builtin/fsck.c b/builtin/fsck.c index 9c8a6d6a8df..32d40d8f9fc 100644 --- a/builtin/fsck.c +++ b/builtin/fsck.c @@ -1,5 +1,6 @@ #define USE_THE_REPOSITORY_VARIABLE #include "builtin.h" +#include "dir.h" #include "gettext.h" #include "hex.h" #include "config.h" diff --git a/builtin/gc.c b/builtin/gc.c index 99431fd4674..b069629676c 100644 --- a/builtin/gc.c +++ b/builtin/gc.c @@ -16,6 +16,7 @@ #include "builtin.h" #include "abspath.h" #include "date.h" +#include "dir.h" #include "environment.h" #include "hex.h" #include "config.h" @@ -28,7 +29,6 @@ #include "commit.h" #include "commit-graph.h" #include "packfile.h" -#include "object-file.h" #include "object-store-ll.h" #include "pack.h" #include "pack-objects.h" diff --git a/builtin/init-db.c b/builtin/init-db.c index 196dccdd77a..39730c1b0ce 100644 --- a/builtin/init-db.c +++ b/builtin/init-db.c @@ -6,9 +6,9 @@ #define USE_THE_REPOSITORY_VARIABLE #include "builtin.h" #include "abspath.h" +#include "dir.h" #include "environment.h" #include "gettext.h" -#include "object-file.h" #include "parse-options.h" #include "path.h" #include "refs.h" diff --git a/builtin/log.c b/builtin/log.c index 0d4c579dad7..06ffaa93e86 100644 --- a/builtin/log.c +++ b/builtin/log.c @@ -10,11 +10,11 @@ #include "builtin.h" #include "abspath.h" #include "config.h" +#include "dir.h" #include "environment.h" #include "gettext.h" #include "hex.h" #include "refs.h" -#include "object-file.h" #include "object-name.h" #include "object-store-ll.h" #include "pager.h" diff --git a/commit-graph.c b/commit-graph.c index 8286d5dda24..3fae20dc21b 100644 --- a/commit-graph.c +++ b/commit-graph.c @@ -4,6 +4,7 @@ #include "git-compat-util.h" #include "config.h" #include "csum-file.h" +#include "dir.h" #include "gettext.h" #include "hex.h" #include "lockfile.h" diff --git a/dir.c b/dir.c index 28b0e03feb4..16ae3b5169d 100644 --- a/dir.c +++ b/dir.c @@ -17,7 +17,6 @@ #include "environment.h" #include "gettext.h" #include "name-hash.h" -#include "object-file.h" #include "object-store-ll.h" #include "path.h" #include "refs.h" @@ -4132,3 +4131,109 @@ int path_match_flags(const char *const str, const enum path_match_flags flags) return is_xplatform_dir_sep(*p); BUG("unreachable"); } + +int mkdir_in_gitdir(const char *path) +{ + if (mkdir(path, 0777)) { + int saved_errno = errno; + struct stat st; + struct strbuf sb = STRBUF_INIT; + + if (errno != EEXIST) + return -1; + /* + * Are we looking at a path in a symlinked worktree + * whose original repository does not yet have it? + * e.g. .git/rr-cache pointing at its original + * repository in which the user hasn't performed any + * conflict resolution yet? + */ + if (lstat(path, &st) || !S_ISLNK(st.st_mode) || + strbuf_readlink(&sb, path, st.st_size) || + !is_absolute_path(sb.buf) || + mkdir(sb.buf, 0777)) { + strbuf_release(&sb); + errno = saved_errno; + return -1; + } + strbuf_release(&sb); + } + return adjust_shared_perm(the_repository, path); +} + +static enum scld_error safe_create_leading_directories_1(char *path, int share) +{ + char *next_component = path + offset_1st_component(path); + enum scld_error ret = SCLD_OK; + + while (ret == SCLD_OK && next_component) { + struct stat st; + char *slash = next_component, slash_character; + + while (*slash && !is_dir_sep(*slash)) + slash++; + + if (!*slash) + break; + + next_component = slash + 1; + while (is_dir_sep(*next_component)) + next_component++; + if (!*next_component) + break; + + slash_character = *slash; + *slash = '\0'; + if (!stat(path, &st)) { + /* path exists */ + if (!S_ISDIR(st.st_mode)) { + errno = ENOTDIR; + ret = SCLD_EXISTS; + } + } else if (mkdir(path, 0777)) { + if (errno == EEXIST && + !stat(path, &st) && S_ISDIR(st.st_mode)) + ; /* somebody created it since we checked */ + else if (errno == ENOENT) + /* + * Either mkdir() failed because + * somebody just pruned the containing + * directory, or stat() failed because + * the file that was in our way was + * just removed. Either way, inform + * the caller that it might be worth + * trying again: + */ + ret = SCLD_VANISHED; + else + ret = SCLD_FAILED; + } else if (share && adjust_shared_perm(the_repository, path)) { + ret = SCLD_PERMS; + } + *slash = slash_character; + } + return ret; +} + +enum scld_error safe_create_leading_directories(char *path) +{ + return safe_create_leading_directories_1(path, 1); +} + +enum scld_error safe_create_leading_directories_no_share(char *path) +{ + return safe_create_leading_directories_1(path, 0); +} + +enum scld_error safe_create_leading_directories_const(const char *path) +{ + int save_errno; + /* path points to cache entries, so xstrdup before messing with it */ + char *buf = xstrdup(path); + enum scld_error result = safe_create_leading_directories(buf); + + save_errno = errno; + free(buf); + errno = save_errno; + return result; +} diff --git a/dir.h b/dir.h index d7e71aa8daa..02c1f9420b0 100644 --- a/dir.h +++ b/dir.h @@ -676,4 +676,39 @@ static inline int starts_with_dot_dot_slash_native(const char *const path) return path_match_flags(path, what | PATH_MATCH_NATIVE); } +/* + * Create the directory containing the named path, using care to be + * somewhat safe against races. Return one of the scld_error values to + * indicate success/failure. On error, set errno to describe the + * problem. + * + * SCLD_VANISHED indicates that one of the ancestor directories of the + * path existed at one point during the function call and then + * suddenly vanished, probably because another process pruned the + * directory while we were working. To be robust against this kind of + * race, callers might want to try invoking the function again when it + * returns SCLD_VANISHED. + * + * safe_create_leading_directories() temporarily changes path while it + * is working but restores it before returning. + * safe_create_leading_directories_const() doesn't modify path, even + * temporarily. Both these variants adjust the permissions of the + * created directories to honor core.sharedRepository, so they are best + * suited for files inside the git dir. For working tree files, use + * safe_create_leading_directories_no_share() instead, as it ignores + * the core.sharedRepository setting. + */ +enum scld_error { + SCLD_OK = 0, + SCLD_FAILED = -1, + SCLD_PERMS = -2, + SCLD_EXISTS = -3, + SCLD_VANISHED = -4 +}; +enum scld_error safe_create_leading_directories(char *path); +enum scld_error safe_create_leading_directories_const(const char *path); +enum scld_error safe_create_leading_directories_no_share(char *path); + +int mkdir_in_gitdir(const char *path); + #endif diff --git a/midx-write.c b/midx-write.c index a628ac24dcb..e01a867c583 100644 --- a/midx-write.c +++ b/midx-write.c @@ -3,6 +3,7 @@ #include "git-compat-util.h" #include "abspath.h" #include "config.h" +#include "dir.h" #include "hex.h" #include "lockfile.h" #include "packfile.h" diff --git a/object-file.c b/object-file.c index 772c311f188..23b2c8560be 100644 --- a/object-file.c +++ b/object-file.c @@ -91,112 +91,6 @@ static int get_conv_flags(unsigned flags) } -int mkdir_in_gitdir(const char *path) -{ - if (mkdir(path, 0777)) { - int saved_errno = errno; - struct stat st; - struct strbuf sb = STRBUF_INIT; - - if (errno != EEXIST) - return -1; - /* - * Are we looking at a path in a symlinked worktree - * whose original repository does not yet have it? - * e.g. .git/rr-cache pointing at its original - * repository in which the user hasn't performed any - * conflict resolution yet? - */ - if (lstat(path, &st) || !S_ISLNK(st.st_mode) || - strbuf_readlink(&sb, path, st.st_size) || - !is_absolute_path(sb.buf) || - mkdir(sb.buf, 0777)) { - strbuf_release(&sb); - errno = saved_errno; - return -1; - } - strbuf_release(&sb); - } - return adjust_shared_perm(the_repository, path); -} - -static enum scld_error safe_create_leading_directories_1(char *path, int share) -{ - char *next_component = path + offset_1st_component(path); - enum scld_error ret = SCLD_OK; - - while (ret == SCLD_OK && next_component) { - struct stat st; - char *slash = next_component, slash_character; - - while (*slash && !is_dir_sep(*slash)) - slash++; - - if (!*slash) - break; - - next_component = slash + 1; - while (is_dir_sep(*next_component)) - next_component++; - if (!*next_component) - break; - - slash_character = *slash; - *slash = '\0'; - if (!stat(path, &st)) { - /* path exists */ - if (!S_ISDIR(st.st_mode)) { - errno = ENOTDIR; - ret = SCLD_EXISTS; - } - } else if (mkdir(path, 0777)) { - if (errno == EEXIST && - !stat(path, &st) && S_ISDIR(st.st_mode)) - ; /* somebody created it since we checked */ - else if (errno == ENOENT) - /* - * Either mkdir() failed because - * somebody just pruned the containing - * directory, or stat() failed because - * the file that was in our way was - * just removed. Either way, inform - * the caller that it might be worth - * trying again: - */ - ret = SCLD_VANISHED; - else - ret = SCLD_FAILED; - } else if (share && adjust_shared_perm(the_repository, path)) { - ret = SCLD_PERMS; - } - *slash = slash_character; - } - return ret; -} - -enum scld_error safe_create_leading_directories(char *path) -{ - return safe_create_leading_directories_1(path, 1); -} - -enum scld_error safe_create_leading_directories_no_share(char *path) -{ - return safe_create_leading_directories_1(path, 0); -} - -enum scld_error safe_create_leading_directories_const(const char *path) -{ - int save_errno; - /* path points to cache entries, so xstrdup before messing with it */ - char *buf = xstrdup(path); - enum scld_error result = safe_create_leading_directories(buf); - - save_errno = errno; - free(buf); - errno = save_errno; - return result; -} - int odb_mkstemp(struct strbuf *temp_filename, const char *pattern) { int fd; diff --git a/object-file.h b/object-file.h index 81b30d269c8..922f2bba8c9 100644 --- a/object-file.h +++ b/object-file.h @@ -21,41 +21,6 @@ extern int fetch_if_missing; int index_fd(struct index_state *istate, struct object_id *oid, int fd, struct stat *st, enum object_type type, const char *path, unsigned flags); int index_path(struct index_state *istate, struct object_id *oid, const char *path, struct stat *st, unsigned flags); -/* - * Create the directory containing the named path, using care to be - * somewhat safe against races. Return one of the scld_error values to - * indicate success/failure. On error, set errno to describe the - * problem. - * - * SCLD_VANISHED indicates that one of the ancestor directories of the - * path existed at one point during the function call and then - * suddenly vanished, probably because another process pruned the - * directory while we were working. To be robust against this kind of - * race, callers might want to try invoking the function again when it - * returns SCLD_VANISHED. - * - * safe_create_leading_directories() temporarily changes path while it - * is working but restores it before returning. - * safe_create_leading_directories_const() doesn't modify path, even - * temporarily. Both these variants adjust the permissions of the - * created directories to honor core.sharedRepository, so they are best - * suited for files inside the git dir. For working tree files, use - * safe_create_leading_directories_no_share() instead, as it ignores - * the core.sharedRepository setting. - */ -enum scld_error { - SCLD_OK = 0, - SCLD_FAILED = -1, - SCLD_PERMS = -2, - SCLD_EXISTS = -3, - SCLD_VANISHED = -4 -}; -enum scld_error safe_create_leading_directories(char *path); -enum scld_error safe_create_leading_directories_const(const char *path); -enum scld_error safe_create_leading_directories_no_share(char *path); - -int mkdir_in_gitdir(const char *path); - int git_open_cloexec(const char *name, int flags); #define git_open(name) git_open_cloexec(name, O_RDONLY) From patchwork Tue Apr 8 10:24:10 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Patrick Steinhardt X-Patchwork-Id: 14042665 Received: from fout-a6-smtp.messagingengine.com (fout-a6-smtp.messagingengine.com [103.168.172.149]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 7CBC92690EC for ; Tue, 8 Apr 2025 10:24:17 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=103.168.172.149 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1744107861; cv=none; b=DC9q1czP3zBOG5oT48FVHQCPHate+hVZS+EdyBkbLk7e2bIuJ0rSRMXQNqIs8oXUtBmoasNTFZdIFZpXuXoOM6z7DiP+E3C5Yj67NsqyoXF7Wf5PfErdTxdSbLNwscjgBeTvNVxUdsqKAf0op7fQZdJDe/14hxsgxsZuY7PaMYg= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1744107861; c=relaxed/simple; bh=nidu1DIQ2Mgi1bhTCaX2P5np7yq+2yqCUD7Oqb5EYrw=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=IeLzlyTCQY/me2Qb3R1v9l+Hi6L/HPsvWoz/B+Hs2HMozTbe+zTsspVFERCrhu4G709yl4l3rv7mqVeWNjgBTUOuBFmat9DwbF29HyDhj6c6KroKi1LvKu+LizVPZBWotYkp0A7nylI5Hz8C10I2TcmzFm++D8R2tSzKdDOmZtE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=pks.im; spf=pass smtp.mailfrom=pks.im; dkim=pass (2048-bit key) header.d=pks.im header.i=@pks.im header.b=Jat/aUDy; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b=VdDTfWNE; arc=none smtp.client-ip=103.168.172.149 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=pks.im Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=pks.im Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=pks.im header.i=@pks.im header.b="Jat/aUDy"; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b="VdDTfWNE" Received: from phl-compute-11.internal (phl-compute-11.phl.internal [10.202.2.51]) by mailfout.phl.internal (Postfix) with ESMTP id 431D81380170 for ; Tue, 8 Apr 2025 06:24:17 -0400 (EDT) Received: from phl-mailfrontend-01 ([10.202.2.162]) by phl-compute-11.internal (MEProxy); Tue, 08 Apr 2025 06:24:17 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=pks.im; h=cc:cc :content-transfer-encoding:content-type:content-type:date:date :from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:subject:subject:to:to; s=fm2; t=1744107857; x=1744194257; bh=xeImIqK4v8f3NSQtc/QrK0m/0HpDRhmrfJ+cjrRH4Ek=; b= Jat/aUDyTrFoZwwxWVma2fQEN/Vih9G69h6sYSM5kD3HwctwIbaLy3nN4Z09fhQs fySNJy8NZZ2vsQKMgzR1SJW4RBL65NBFdmhdK3Ic5cyeVywAGLM7hejw+ttaAfJk l6WKhP//GX7d2wUO6RUFSMUGsUxk4IX3ZT92jwrq5A1sv8OcQlCMTBNceuDh8yDs M/vYdxUyPtQUa8w2jYNpFkNDSHwrRkS9Zdi10yG+Ca4PPHKi7HeiNfsAo+IDkj5Y RpQLPIpWgerGBrWfxchIjTf2hgRF8TDEN40yJJ5M6lrPStnWIQwcnJXEr9bWi94K Z8PwbmQuuR6TTgk29C5uHg== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-transfer-encoding :content-type:content-type:date:date:feedback-id:feedback-id :from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:subject:subject:to:to:x-me-proxy :x-me-sender:x-me-sender:x-sasl-enc; s=fm2; t=1744107857; x= 1744194257; bh=xeImIqK4v8f3NSQtc/QrK0m/0HpDRhmrfJ+cjrRH4Ek=; b=V dDTfWNEYCcg2/zYbJpzDwIx07FkuYQ6Uhpngqqty65dUCgg5jIverVYlH4cvcKpC V4qhXOHSVhLO++DFu+ZT5Eun0ATbNf/uPfpKkk17hijq6TTMk2yRepyKHbpPTYP4 mKQ1cR/24r7SmelSu5Scje8WaO1XwEL1u3kYORTNw06u+bRB4JX6nB1agxFn0A9Y wJypJR+S28lxxW9e7Gu6X/hc1UIzs/Q9xFdAn3uU80j5a8qABETiO/Aeq8eVRBpj ZxuqEflkwg7Ctmrp1ljABzLC5f+9+G+f6MxM62JrgqQBfRU3DwdwECUTGgr5ZCvV VRGlKiFyMxmMfRzv541hg== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeefvddrtddtgddvtddvkeegucetufdoteggodetrf dotffvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdggtfgfnhhsuhgsshgtrhhisggv pdfurfetoffkrfgpnffqhgenuceurghilhhouhhtmecufedttdenucenucfjughrpefhff fugggtgffkfhgjvfevofesthejredtredtjeenucfhrhhomheprfgrthhrihgtkhcuufht vghinhhhrghrughtuceophhssehpkhhsrdhimheqnecuggftrfgrthhtvghrnhepffeuie dujedvkeehuedvkeefffeivdeuleetkeduheejteekgedvudfgtdfgieelnecuvehluhhs thgvrhfuihiivgeptdenucfrrghrrghmpehmrghilhhfrhhomhepphhssehpkhhsrdhimh dpnhgspghrtghpthhtohepuddpmhhouggvpehsmhhtphhouhhtpdhrtghpthhtohepghhi thesvhhgvghrrdhkvghrnhgvlhdrohhrgh X-ME-Proxy: Feedback-ID: i197146af:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA for ; Tue, 8 Apr 2025 06:24:16 -0400 (EDT) Received: by vm-mail (OpenSMTPD) with ESMTPSA id f2dc84f9 (TLSv1.3:TLS_AES_256_GCM_SHA384:256:NO) for ; Tue, 8 Apr 2025 10:24:14 +0000 (UTC) From: Patrick Steinhardt Date: Tue, 08 Apr 2025 12:24:10 +0200 Subject: [PATCH 2/9] object-file: move `git_open_cloexec()` to "compat/open.c" Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20250408-pks-split-object-file-v1-2-f1fd50191143@pks.im> References: <20250408-pks-split-object-file-v1-0-f1fd50191143@pks.im> In-Reply-To: <20250408-pks-split-object-file-v1-0-f1fd50191143@pks.im> To: git@vger.kernel.org Cc: X-Mailer: b4 0.14.2 The `git_open_cloexec()` wrapper function provides the ability to open a file with `O_CLOEXEC` in a platform-agnostic way. This function is provided by "object-file.c" even though it is not specific to the object subsystem at all. Move the file into "compat/open.c". This file already exists before this commit, but has only been compiled conditionally depending on whether or not open(3p) may return EINTR. With this change we now unconditionally compile the object, but wrap `git_open_with_retry()` in an ifdef. Signed-off-by: Patrick Steinhardt --- Makefile | 2 +- commit-graph.c | 1 - compat/open.c | 29 +++++++++++++++++++++++++++++ git-compat-util.h | 3 +++ meson.build | 1 + midx.c | 1 - object-file.c | 27 --------------------------- object-file.h | 3 --- pack-bitmap.c | 1 - pack-mtimes.c | 1 - pack-revindex.c | 1 - 11 files changed, 34 insertions(+), 36 deletions(-) diff --git a/Makefile b/Makefile index c41fc41ef0e..bb5407b4703 100644 --- a/Makefile +++ b/Makefile @@ -994,6 +994,7 @@ LIB_OBJS += common-exit.o LIB_OBJS += common-init.o LIB_OBJS += compat/nonblock.o LIB_OBJS += compat/obstack.o +LIB_OBJS += compat/open.o LIB_OBJS += compat/terminal.o LIB_OBJS += compiler-tricks/not-constant.o LIB_OBJS += config.o @@ -1812,7 +1813,6 @@ ifdef FREAD_READS_DIRECTORIES endif ifdef OPEN_RETURNS_EINTR COMPAT_CFLAGS += -DOPEN_RETURNS_EINTR - COMPAT_OBJS += compat/open.o endif ifdef NO_SYMLINK_HEAD BASIC_CFLAGS += -DNO_SYMLINK_HEAD diff --git a/commit-graph.c b/commit-graph.c index 3fae20dc21b..8060c358b84 100644 --- a/commit-graph.c +++ b/commit-graph.c @@ -14,7 +14,6 @@ #include "refs.h" #include "hash-lookup.h" #include "commit-graph.h" -#include "object-file.h" #include "object-store-ll.h" #include "oid-array.h" #include "path.h" diff --git a/compat/open.c b/compat/open.c index eb3754a23b8..37ae2b1aeb9 100644 --- a/compat/open.c +++ b/compat/open.c @@ -1,5 +1,6 @@ #include "git-compat-util.h" +#ifdef OPEN_RETURNS_EINTR #undef open int git_open_with_retry(const char *path, int flags, ...) { @@ -23,3 +24,31 @@ int git_open_with_retry(const char *path, int flags, ...) return ret; } +#endif + +int git_open_cloexec(const char *name, int flags) +{ + int fd; + static int o_cloexec = O_CLOEXEC; + + fd = open(name, flags | o_cloexec); + if ((o_cloexec & O_CLOEXEC) && fd < 0 && errno == EINVAL) { + /* Try again w/o O_CLOEXEC: the kernel might not support it */ + o_cloexec &= ~O_CLOEXEC; + fd = open(name, flags | o_cloexec); + } + +#if defined(F_GETFD) && defined(F_SETFD) && defined(FD_CLOEXEC) + { + static int fd_cloexec = FD_CLOEXEC; + + if (!o_cloexec && 0 <= fd && fd_cloexec) { + /* Opened w/o O_CLOEXEC? try with fcntl(2) to add it */ + int flags = fcntl(fd, F_GETFD); + if (fcntl(fd, F_SETFD, flags | fd_cloexec)) + fd_cloexec = 0; + } + } +#endif + return fd; +} diff --git a/git-compat-util.h b/git-compat-util.h index cf733b38acd..9273a8ee087 100644 --- a/git-compat-util.h +++ b/git-compat-util.h @@ -1000,6 +1000,9 @@ int git_vsnprintf(char *str, size_t maxsize, int git_open_with_retry(const char *path, int flag, ...); #endif +int git_open_cloexec(const char *name, int flags); +#define git_open(name) git_open_cloexec(name, O_RDONLY) + #ifdef __GLIBC_PREREQ #if __GLIBC_PREREQ(2, 1) #define HAVE_STRCHRNUL diff --git a/meson.build b/meson.build index 145d2f7ff9e..a55e800b85b 100644 --- a/meson.build +++ b/meson.build @@ -263,6 +263,7 @@ libgit_sources = [ 'common-init.c', 'compat/nonblock.c', 'compat/obstack.c', + 'compat/open.c', 'compat/terminal.c', 'compiler-tricks/not-constant.c', 'config.c', diff --git a/midx.c b/midx.c index 807fdf72f7b..3d0015f7828 100644 --- a/midx.c +++ b/midx.c @@ -5,7 +5,6 @@ #include "dir.h" #include "hex.h" #include "packfile.h" -#include "object-file.h" #include "hash-lookup.h" #include "midx.h" #include "progress.h" diff --git a/object-file.c b/object-file.c index 23b2c8560be..1a20c7fa072 100644 --- a/object-file.c +++ b/object-file.c @@ -834,33 +834,6 @@ int stream_object_signature(struct repository *r, const struct object_id *oid) return !oideq(oid, &real_oid) ? -1 : 0; } -int git_open_cloexec(const char *name, int flags) -{ - int fd; - static int o_cloexec = O_CLOEXEC; - - fd = open(name, flags | o_cloexec); - if ((o_cloexec & O_CLOEXEC) && fd < 0 && errno == EINVAL) { - /* Try again w/o O_CLOEXEC: the kernel might not support it */ - o_cloexec &= ~O_CLOEXEC; - fd = open(name, flags | o_cloexec); - } - -#if defined(F_GETFD) && defined(F_SETFD) && defined(FD_CLOEXEC) - { - static int fd_cloexec = FD_CLOEXEC; - - if (!o_cloexec && 0 <= fd && fd_cloexec) { - /* Opened w/o O_CLOEXEC? try with fcntl(2) to add it */ - int flags = fcntl(fd, F_GETFD); - if (fcntl(fd, F_SETFD, flags | fd_cloexec)) - fd_cloexec = 0; - } - } -#endif - return fd; -} - /* * Find "oid" as a loose object in the local repository or in an alternate. * Returns 0 on success, negative on failure. diff --git a/object-file.h b/object-file.h index 922f2bba8c9..353d8a85c33 100644 --- a/object-file.h +++ b/object-file.h @@ -21,9 +21,6 @@ extern int fetch_if_missing; int index_fd(struct index_state *istate, struct object_id *oid, int fd, struct stat *st, enum object_type type, const char *path, unsigned flags); int index_path(struct index_state *istate, struct object_id *oid, const char *path, struct stat *st, unsigned flags); -int git_open_cloexec(const char *name, int flags); -#define git_open(name) git_open_cloexec(name, O_RDONLY) - /** * unpack_loose_header() initializes the data stream needed to unpack * a loose object header. diff --git a/pack-bitmap.c b/pack-bitmap.c index 7fd78c634ef..0dbd7c4ffe1 100644 --- a/pack-bitmap.c +++ b/pack-bitmap.c @@ -17,7 +17,6 @@ #include "packfile.h" #include "repository.h" #include "trace2.h" -#include "object-file.h" #include "object-store-ll.h" #include "list-objects-filter-options.h" #include "midx.h" diff --git a/pack-mtimes.c b/pack-mtimes.c index cdf30b8d2b0..bcea28e521d 100644 --- a/pack-mtimes.c +++ b/pack-mtimes.c @@ -1,7 +1,6 @@ #include "git-compat-util.h" #include "gettext.h" #include "pack-mtimes.h" -#include "object-file.h" #include "object-store-ll.h" #include "packfile.h" #include "strbuf.h" diff --git a/pack-revindex.c b/pack-revindex.c index 038e0c96b1c..1ee7b49e206 100644 --- a/pack-revindex.c +++ b/pack-revindex.c @@ -1,7 +1,6 @@ #include "git-compat-util.h" #include "gettext.h" #include "pack-revindex.h" -#include "object-file.h" #include "object-store-ll.h" #include "packfile.h" #include "strbuf.h" From patchwork Tue Apr 8 10:24:11 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Patrick Steinhardt X-Patchwork-Id: 14042663 Received: from fhigh-a1-smtp.messagingengine.com (fhigh-a1-smtp.messagingengine.com [103.168.172.152]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 809BE2690F1 for ; Tue, 8 Apr 2025 10:24:18 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=103.168.172.152 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1744107860; cv=none; b=mK8Gh33floFhskjjy6tk9RVe7TRE72NjkV3AgR4ExyUR4r1M4/N2s//Pj6H6sBVTa7ZvQyiI2jZ0bCJpvcAWNAnGcYZGcmiZS/NZkSyZmtftcFRMWGyBZ9rvsvwMWrJyhJ4MTwnBqqipZMQgk/qTNjpveuC+r28iV+SL7DKftBk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1744107860; c=relaxed/simple; bh=Gskf+OAr0VPgd6DysmZ5QVhDL63xRF/SGIWBMbm5czA=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=FKLbF01pt9v/f56rerU31E/JItkSk9PTk+Zp1/UQFvjOp/FpdD3vROKINXHRQ6M6adeE0pOCKL4wKV4imMB4k2GW8DzD8BJ4sN5wnl+93TVkLc58xzi3eeRFZH+cVXxEopqIz+EtdC5Q0hEOkv3wYAUxO9MUtjV+A4c2RvHppm4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=pks.im; spf=pass smtp.mailfrom=pks.im; dkim=pass (2048-bit key) header.d=pks.im header.i=@pks.im header.b=FljR9W3+; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b=xdjtLe0f; arc=none smtp.client-ip=103.168.172.152 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=pks.im Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=pks.im Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=pks.im header.i=@pks.im header.b="FljR9W3+"; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b="xdjtLe0f" Received: from phl-compute-01.internal (phl-compute-01.phl.internal [10.202.2.41]) by mailfhigh.phl.internal (Postfix) with ESMTP id 88384114011C for ; Tue, 8 Apr 2025 06:24:17 -0400 (EDT) Received: from phl-mailfrontend-02 ([10.202.2.163]) by phl-compute-01.internal (MEProxy); Tue, 08 Apr 2025 06:24:17 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=pks.im; h=cc:cc :content-transfer-encoding:content-type:content-type:date:date :from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:subject:subject:to:to; s=fm2; t=1744107857; x=1744194257; bh=6B7hrXwHVmBXGx54iYdPAxQGgwq8NB66MUMbUqAz67U=; b= FljR9W3+cRu10/OHIuQpGv1U8zPpUMOxnj9+VjELAjBPZn3tCy1fpv5J/Pux0mQ3 bDEr0NTv96x6mS2+7MirbeDEecZNdcxdk9TTmQydjzrrvh/s28Dx+rg4YE6tFN+5 uHpuJHDyOIPzbXd6QLrfibrrIdMPJ9x3Gga0hrA+OqmsO5l8IOEletNsB2LxwwLq 4ZRSfXCZirgWf+Xgn6vOU5ovcJWFKLk5U+gNC3I9RLxdxsz4OvSkGskdxiJuFF6G CGf4GpYOQjjYxM3Rg4frzxa7atP+xkyp4NRms1NRw5etc16H+2bWqUaUbYU8XXQ7 5M9wWuz9YT4GXfNX4QpoWg== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-transfer-encoding :content-type:content-type:date:date:feedback-id:feedback-id :from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:subject:subject:to:to:x-me-proxy :x-me-sender:x-me-sender:x-sasl-enc; s=fm2; t=1744107857; x= 1744194257; bh=6B7hrXwHVmBXGx54iYdPAxQGgwq8NB66MUMbUqAz67U=; b=x djtLe0ftmO9yg6mLSQ1ZOwuVHm8Uw53MklQ8bZrNjY/W1pizjbssP+szSuaIcf3Z BWtaFDxjAdo+VE2Fh0VvuLhgqA1mB3XGMzivHcVtRRn/Fsu7JaYpElYgneuVbueM fBG5jd1JiYAY+lWdlQxIyLUoC0jGeVCcNWP6w26lMeBdF83gwUHPqZUqSfeAS8NT vp8CH6GFLZ7zRuH2RUwnNVisy6XT+O61Q2HCB33XLaNoR15iQPjCISTyoPqqZPhU A+Sv+c/63U4Bc8lDfi2oWmxmjVLW2Z8FD4RZkpzQ6qZOq9OwDA+rIJ5ndmYfh1Ii A1ycztmwqZE/XfZoCHYkQ== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeefvddrtddtgddvtddvkeegucetufdoteggodetrf dotffvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdggtfgfnhhsuhgsshgtrhhisggv pdfurfetoffkrfgpnffqhgenuceurghilhhouhhtmecufedttdenucenucfjughrpefhff fugggtgffkfhgjvfevofesthejredtredtjeenucfhrhhomheprfgrthhrihgtkhcuufht vghinhhhrghrughtuceophhssehpkhhsrdhimheqnecuggftrfgrthhtvghrnhepffeuie dujedvkeehuedvkeefffeivdeuleetkeduheejteekgedvudfgtdfgieelnecuvehluhhs thgvrhfuihiivgeptdenucfrrghrrghmpehmrghilhhfrhhomhepphhssehpkhhsrdhimh dpnhgspghrtghpthhtohepuddpmhhouggvpehsmhhtphhouhhtpdhrtghpthhtohepghhi thesvhhgvghrrdhkvghrnhgvlhdrohhrgh X-ME-Proxy: Feedback-ID: i197146af:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA for ; Tue, 8 Apr 2025 06:24:16 -0400 (EDT) Received: by vm-mail (OpenSMTPD) with ESMTPSA id 169010ea (TLSv1.3:TLS_AES_256_GCM_SHA384:256:NO) for ; Tue, 8 Apr 2025 10:24:15 +0000 (UTC) From: Patrick Steinhardt Date: Tue, 08 Apr 2025 12:24:11 +0200 Subject: [PATCH 3/9] object-file: move `xmmap()` into "wrapper.c" Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20250408-pks-split-object-file-v1-3-f1fd50191143@pks.im> References: <20250408-pks-split-object-file-v1-0-f1fd50191143@pks.im> In-Reply-To: <20250408-pks-split-object-file-v1-0-f1fd50191143@pks.im> To: git@vger.kernel.org Cc: X-Mailer: b4 0.14.2 The `xmmap()` function is provided by "object-file.c" even though its functionality got nothing to do with the object file subsystem. Move it into "wrapper.c", whose header already declares those functions. Signed-off-by: Patrick Steinhardt --- object-file.c | 48 ------------------------------------------------ wrapper.c | 48 ++++++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 48 insertions(+), 48 deletions(-) diff --git a/object-file.c b/object-file.c index 1a20c7fa072..ea2ed7628e6 100644 --- a/object-file.c +++ b/object-file.c @@ -719,54 +719,6 @@ int has_loose_object(const struct object_id *oid) return check_and_freshen(oid, 0); } -static void mmap_limit_check(size_t length) -{ - static size_t limit = 0; - if (!limit) { - limit = git_env_ulong("GIT_MMAP_LIMIT", 0); - if (!limit) - limit = SIZE_MAX; - } - if (length > limit) - die(_("attempting to mmap %"PRIuMAX" over limit %"PRIuMAX), - (uintmax_t)length, (uintmax_t)limit); -} - -void *xmmap_gently(void *start, size_t length, - int prot, int flags, int fd, off_t offset) -{ - void *ret; - - mmap_limit_check(length); - ret = mmap(start, length, prot, flags, fd, offset); - if (ret == MAP_FAILED && !length) - ret = NULL; - return ret; -} - -const char *mmap_os_err(void) -{ - static const char blank[] = ""; -#if defined(__linux__) - if (errno == ENOMEM) { - /* this continues an existing error message: */ - static const char enomem[] = -", check sys.vm.max_map_count and/or RLIMIT_DATA"; - return enomem; - } -#endif /* OS-specific bits */ - return blank; -} - -void *xmmap(void *start, size_t length, - int prot, int flags, int fd, off_t offset) -{ - void *ret = xmmap_gently(start, length, prot, flags, fd, offset); - if (ret == MAP_FAILED) - die_errno(_("mmap failed%s"), mmap_os_err()); - return ret; -} - static int format_object_header_literally(char *str, size_t size, const char *type, size_t objsize) { diff --git a/wrapper.c b/wrapper.c index 8b985931490..3c79778055e 100644 --- a/wrapper.c +++ b/wrapper.c @@ -829,3 +829,51 @@ uint32_t git_rand(unsigned flags) return result; } + +static void mmap_limit_check(size_t length) +{ + static size_t limit = 0; + if (!limit) { + limit = git_env_ulong("GIT_MMAP_LIMIT", 0); + if (!limit) + limit = SIZE_MAX; + } + if (length > limit) + die(_("attempting to mmap %"PRIuMAX" over limit %"PRIuMAX), + (uintmax_t)length, (uintmax_t)limit); +} + +void *xmmap_gently(void *start, size_t length, + int prot, int flags, int fd, off_t offset) +{ + void *ret; + + mmap_limit_check(length); + ret = mmap(start, length, prot, flags, fd, offset); + if (ret == MAP_FAILED && !length) + ret = NULL; + return ret; +} + +const char *mmap_os_err(void) +{ + static const char blank[] = ""; +#if defined(__linux__) + if (errno == ENOMEM) { + /* this continues an existing error message: */ + static const char enomem[] = +", check sys.vm.max_map_count and/or RLIMIT_DATA"; + return enomem; + } +#endif /* OS-specific bits */ + return blank; +} + +void *xmmap(void *start, size_t length, + int prot, int flags, int fd, off_t offset) +{ + void *ret = xmmap_gently(start, length, prot, flags, fd, offset); + if (ret == MAP_FAILED) + die_errno(_("mmap failed%s"), mmap_os_err()); + return ret; +} From patchwork Tue Apr 8 10:24:12 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Patrick Steinhardt X-Patchwork-Id: 14042669 Received: from fout-a6-smtp.messagingengine.com (fout-a6-smtp.messagingengine.com [103.168.172.149]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 3774D2690EB for ; Tue, 8 Apr 2025 10:24:18 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=103.168.172.149 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1744107863; cv=none; b=QCra76pU5STwsYLMZWkgI0PPo4HVAxpTzDM52g7OhmAHDA4Fbl8dWuFSUaUwwb8ssduRbdOumbf1Gk83MjTTX1H3A0OYBayKDvHaaj9D3kJ+sV5+1e6H0W4q8rH6SadCrDA/Nq8Ull1UMD8iO83BEhTzylgOduyD1WgJJ5PLU28= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1744107863; c=relaxed/simple; bh=uMl4LviY7WqPqayCzsMbV+B7Fxz0Mu4KpfJGkzijluc=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=DK0eRT5sL3IkZ3K24CpWU8iSkTUZfyU1twQbPzvS+T3hdLO8YQMGtaeA8EHdXL4u4cNbO0HDHjNZkux1naoXReIWCpmRIps32vSAhWCJx/ItcthB4Q40k4MTOQZQai/f6yvLvHKmohTT4KnWs5l2Y8aeHlM+zUPAMuNqbsrBkBU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=pks.im; spf=pass smtp.mailfrom=pks.im; dkim=pass (2048-bit key) header.d=pks.im header.i=@pks.im header.b=KqLNpZu5; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b=rRhe5ykw; arc=none smtp.client-ip=103.168.172.149 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=pks.im Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=pks.im Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=pks.im header.i=@pks.im header.b="KqLNpZu5"; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b="rRhe5ykw" Received: from phl-compute-02.internal (phl-compute-02.phl.internal [10.202.2.42]) by mailfout.phl.internal (Postfix) with ESMTP id 2E24E138008E for ; Tue, 8 Apr 2025 06:24:18 -0400 (EDT) Received: from phl-mailfrontend-01 ([10.202.2.162]) by phl-compute-02.internal (MEProxy); Tue, 08 Apr 2025 06:24:18 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=pks.im; h=cc:cc :content-transfer-encoding:content-type:content-type:date:date :from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:subject:subject:to:to; s=fm2; t=1744107858; x=1744194258; bh=eGs4n3sENsnuOj2G7WIHVUjDXM9noKZHBWBbVVY/kIU=; b= KqLNpZu5g/jvVbAliAIbv2RGpPaUS98vGtWIwPPYO/Ekq9UKftp+gZTWo0yr/b9J YOLPdl5VDZG44eL3HuXx9BUgQLbshsnhrUMsBuAU3KT4twkyEJMdbjAIK3z+Xfk7 6tAfgvwA7A/yEllXwPTqK8l0/8WblnW1E3KWSb3EmSSsts5Zp+YaIlmlKhbnoElf ka0T00cVZ5K2igyRE6J47wkYeSwgzDc5j1ME7lcNrgHrGbwF7ZCPwVwu9dIU6ueF 26uWHopHIiUbJ97KM/g7Yjiq4c4qfwWvypGVQvTN2aE3tF9xl5KgSAYa/mAzX0Cc TrtIHVddWd6rrNaTlJqSOA== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-transfer-encoding :content-type:content-type:date:date:feedback-id:feedback-id :from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:subject:subject:to:to:x-me-proxy :x-me-sender:x-me-sender:x-sasl-enc; s=fm2; t=1744107858; x= 1744194258; bh=eGs4n3sENsnuOj2G7WIHVUjDXM9noKZHBWBbVVY/kIU=; b=r Rhe5ykwAod83gPiDjGhBDnOnNGBWy+JJXcUHAEq+rd+MLjL1+hTGoppdiSgOB6YS pT80XzeWxErLEVWVeXivbSjk8WPy/l8Wr2pCOOjQ6FVxZcFXlQDtJV7WGrXSnjck +Q84nz5My3HTyYhHptrCFplUoFL67sX8LaogG+MypRA49YREL5r4EmAJeu0SAGLa IUFerzWjK54naUC1PT4nRUPavO/MBUqY8fC5So2YI5mcvLWqHJJg0PHL84uZL0w2 UOai0P8F8jFN4KBAk0m+hlXd8ucod/FnvZV9lmX2ezk9iCuWuR7n7XMd5gw4lMt2 t47nWqd/EgQW24EpkdM8g== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeefvddrtddtgddvtddvkeegucetufdoteggodetrf dotffvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdggtfgfnhhsuhgsshgtrhhisggv pdfurfetoffkrfgpnffqhgenuceurghilhhouhhtmecufedttdenucgovfgvgihtqfhnlh ihqddqteefjeefqddtgeculdehtddmnecujfgurhephfffufggtgfgkfhfjgfvvefoseht jeertdertdejnecuhfhrohhmpefrrghtrhhitghkucfuthgvihhnhhgrrhguthcuoehpsh esphhkshdrihhmqeenucggtffrrghtthgvrhhnpeefueevuedvuedugeeiveettdduheel gfelvdduveeljeetfeejjeefkefhhedtueenucffohhmrghinheptgefqdhmvghsohhnrd gsuhhilhgupdhprggtkhgvugdrihhsnecuvehluhhsthgvrhfuihiivgeptdenucfrrghr rghmpehmrghilhhfrhhomhepphhssehpkhhsrdhimhdpnhgspghrtghpthhtohepuddpmh houggvpehsmhhtphhouhhtpdhrtghpthhtohepghhithesvhhgvghrrdhkvghrnhgvlhdr ohhrgh X-ME-Proxy: Feedback-ID: i197146af:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA for ; Tue, 8 Apr 2025 06:24:17 -0400 (EDT) Received: by vm-mail (OpenSMTPD) with ESMTPSA id 9033776c (TLSv1.3:TLS_AES_256_GCM_SHA384:256:NO) for ; Tue, 8 Apr 2025 10:24:15 +0000 (UTC) From: Patrick Steinhardt Date: Tue, 08 Apr 2025 12:24:12 +0200 Subject: [PATCH 4/9] object-file: split out functions relating to object store subsystem Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20250408-pks-split-object-file-v1-4-f1fd50191143@pks.im> References: <20250408-pks-split-object-file-v1-0-f1fd50191143@pks.im> In-Reply-To: <20250408-pks-split-object-file-v1-0-f1fd50191143@pks.im> To: git@vger.kernel.org Cc: X-Mailer: b4 0.14.2 While we have the "object-store.h" header, most of the functionality for object stores is actually hosted in "object-file.c". This makes it hard to find relevant functions and causes us to mix up concerns. Split out functions relating to the object store subsystem into a new "object-store.c" file. Signed-off-by: Patrick Steinhardt --- Makefile | 1 + builtin/checkout.c | 1 + builtin/merge-file.c | 1 + builtin/mktree.c | 1 + builtin/notes.c | 1 + builtin/receive-pack.c | 1 + builtin/tag.c | 1 + builtin/unpack-file.c | 1 + builtin/unpack-objects.c | 1 + commit.c | 1 + http-push.c | 1 + match-trees.c | 3 +- merge-ort.c | 3 +- meson.build | 1 + notes-cache.c | 3 +- notes.c | 3 +- object-file.c | 991 +---------------------------------------------- object-file.h | 68 +++- object-store-ll.h | 60 +-- object-store.c | 972 ++++++++++++++++++++++++++++++++++++++++++++++ 20 files changed, 1074 insertions(+), 1041 deletions(-) diff --git a/Makefile b/Makefile index bb5407b4703..d8ad76fb791 100644 --- a/Makefile +++ b/Makefile @@ -1086,6 +1086,7 @@ LIB_OBJS += notes.o LIB_OBJS += object-file-convert.o LIB_OBJS += object-file.o LIB_OBJS += object-name.o +LIB_OBJS += object-store.o LIB_OBJS += object.o LIB_OBJS += oid-array.o LIB_OBJS += oidmap.o diff --git a/builtin/checkout.c b/builtin/checkout.c index 2e7486cf658..3e68623838a 100644 --- a/builtin/checkout.c +++ b/builtin/checkout.c @@ -18,6 +18,7 @@ #include "lockfile.h" #include "mem-pool.h" #include "merge-recursive.h" +#include "object-file.h" #include "object-name.h" #include "object-store-ll.h" #include "parse-options.h" diff --git a/builtin/merge-file.c b/builtin/merge-file.c index 7e315f374b2..2b16b10d2ca 100644 --- a/builtin/merge-file.c +++ b/builtin/merge-file.c @@ -5,6 +5,7 @@ #include "abspath.h" #include "diff.h" #include "hex.h" +#include "object-file.h" #include "object-name.h" #include "object-store.h" #include "config.h" diff --git a/builtin/mktree.c b/builtin/mktree.c index 3c16faa40e3..0644f951161 100644 --- a/builtin/mktree.c +++ b/builtin/mktree.c @@ -11,6 +11,7 @@ #include "strbuf.h" #include "tree.h" #include "parse-options.h" +#include "object-file.h" #include "object-store-ll.h" static struct treeent { diff --git a/builtin/notes.c b/builtin/notes.c index ff61ec5f2da..0dbc233752d 100644 --- a/builtin/notes.c +++ b/builtin/notes.c @@ -14,6 +14,7 @@ #include "gettext.h" #include "hex.h" #include "notes.h" +#include "object-file.h" #include "object-name.h" #include "object-store-ll.h" #include "path.h" diff --git a/builtin/receive-pack.c b/builtin/receive-pack.c index b3e2a9d0c60..ee51bd76f60 100644 --- a/builtin/receive-pack.c +++ b/builtin/receive-pack.c @@ -31,6 +31,7 @@ #include "tmp-objdir.h" #include "oidset.h" #include "packfile.h" +#include "object-file.h" #include "object-name.h" #include "object-store-ll.h" #include "path.h" diff --git a/builtin/tag.c b/builtin/tag.c index 7c173535cb3..536a01ff3ae 100644 --- a/builtin/tag.c +++ b/builtin/tag.c @@ -17,6 +17,7 @@ #include "gettext.h" #include "hex.h" #include "refs.h" +#include "object-file.h" #include "object-name.h" #include "object-store-ll.h" #include "path.h" diff --git a/builtin/unpack-file.c b/builtin/unpack-file.c index fb5fcbc40a8..b19e5cabd03 100644 --- a/builtin/unpack-file.c +++ b/builtin/unpack-file.c @@ -2,6 +2,7 @@ #include "builtin.h" #include "config.h" #include "hex.h" +#include "object-file.h" #include "object-name.h" #include "object-store-ll.h" diff --git a/builtin/unpack-objects.c b/builtin/unpack-objects.c index 3bbcaf2de9b..4078eab9252 100644 --- a/builtin/unpack-objects.c +++ b/builtin/unpack-objects.c @@ -8,6 +8,7 @@ #include "gettext.h" #include "git-zlib.h" #include "hex.h" +#include "object-file.h" #include "object-store-ll.h" #include "object.h" #include "delta.h" diff --git a/commit.c b/commit.c index 48aeefaad31..fbf4f8e87fd 100644 --- a/commit.c +++ b/commit.c @@ -29,6 +29,7 @@ #include "tree.h" #include "hook.h" #include "parse.h" +#include "object-file.h" #include "object-file-convert.h" static struct commit_extra_header *read_commit_extra_header_lines(const char *buf, size_t len, const char **); diff --git a/http-push.c b/http-push.c index 1b030d96f48..806eb67cf1b 100644 --- a/http-push.c +++ b/http-push.c @@ -19,6 +19,7 @@ #include "tree-walk.h" #include "url.h" #include "packfile.h" +#include "object-file.h" #include "object-store-ll.h" #include "commit-reach.h" diff --git a/match-trees.c b/match-trees.c index ef14ceb594c..72922d5d64e 100644 --- a/match-trees.c +++ b/match-trees.c @@ -6,7 +6,8 @@ #include "strbuf.h" #include "tree.h" #include "tree-walk.h" -#include "object-store-ll.h" +#include "object-file.h" +#include "object-store.h" #include "repository.h" static int score_missing(unsigned mode) diff --git a/merge-ort.c b/merge-ort.c index 1554900d984..5de3e2fc159 100644 --- a/merge-ort.c +++ b/merge-ort.c @@ -36,8 +36,9 @@ #include "merge-ll.h" #include "match-trees.h" #include "mem-pool.h" +#include "object-file.h" #include "object-name.h" -#include "object-store-ll.h" +#include "object-store.h" #include "oid-array.h" #include "path.h" #include "promisor-remote.h" diff --git a/meson.build b/meson.build index a55e800b85b..f656f978020 100644 --- a/meson.build +++ b/meson.build @@ -355,6 +355,7 @@ libgit_sources = [ 'object-file-convert.c', 'object-file.c', 'object-name.c', + 'object-store.c', 'object.c', 'oid-array.c', 'oidmap.c', diff --git a/notes-cache.c b/notes-cache.c index ecfdf6e43b5..150241b15e0 100644 --- a/notes-cache.c +++ b/notes-cache.c @@ -2,7 +2,8 @@ #include "git-compat-util.h" #include "notes-cache.h" -#include "object-store-ll.h" +#include "object-file.h" +#include "object-store.h" #include "pretty.h" #include "repository.h" #include "commit.h" diff --git a/notes.c b/notes.c index ce5a1006a83..d9645c4b5dc 100644 --- a/notes.c +++ b/notes.c @@ -6,8 +6,9 @@ #include "environment.h" #include "hex.h" #include "notes.h" +#include "object-file.h" #include "object-name.h" -#include "object-store-ll.h" +#include "object-store.h" #include "utf8.h" #include "strbuf.h" #include "tree-walk.h" diff --git a/object-file.c b/object-file.c index ea2ed7628e6..1c74aa239fb 100644 --- a/object-file.c +++ b/object-file.c @@ -11,75 +11,26 @@ #define DISABLE_SIGN_COMPARE_WARNINGS #include "git-compat-util.h" -#include "abspath.h" -#include "config.h" +#include "bulk-checkin.h" #include "convert.h" #include "environment.h" +#include "fsck.h" #include "gettext.h" #include "hex.h" -#include "string-list.h" -#include "lockfile.h" -#include "pack.h" -#include "commit.h" -#include "run-command.h" -#include "refs.h" -#include "bulk-checkin.h" -#include "repository.h" -#include "replace-object.h" -#include "streaming.h" -#include "dir.h" -#include "list.h" -#include "quote.h" -#include "packfile.h" +#include "loose.h" +#include "object-file-convert.h" #include "object-file.h" #include "object-store.h" #include "oidtree.h" +#include "pack.h" +#include "packfile.h" #include "path.h" -#include "promisor-remote.h" #include "setup.h" -#include "submodule.h" -#include "fsck.h" -#include "loose.h" -#include "object-file-convert.h" +#include "streaming.h" /* The maximum size for an object header. */ #define MAX_HEADER_LEN 32 -/* - * This is meant to hold a *small* number of objects that you would - * want repo_read_object_file() to be able to return, but yet you do not want - * to write them into the object store (e.g. a browse-only - * application). - */ -static struct cached_object_entry { - struct object_id oid; - struct cached_object { - enum object_type type; - const void *buf; - unsigned long size; - } value; -} *cached_objects; -static int cached_object_nr, cached_object_alloc; - -static const struct cached_object *find_cached_object(const struct object_id *oid) -{ - static const struct cached_object empty_tree = { - .type = OBJ_TREE, - .buf = "", - }; - int i; - const struct cached_object_entry *co = cached_objects; - - for (i = 0; i < cached_object_nr; i++, co++) { - if (oideq(&co->oid, oid)) - return &co->value; - } - if (oideq(oid, the_hash_algo->empty_tree)) - return &empty_tree; - return NULL; -} - - static int get_conv_flags(unsigned flags) { if (flags & HASH_RENORMALIZE) @@ -90,40 +41,6 @@ static int get_conv_flags(unsigned flags) return 0; } - -int odb_mkstemp(struct strbuf *temp_filename, const char *pattern) -{ - int fd; - /* - * we let the umask do its job, don't try to be more - * restrictive except to remove write permission. - */ - int mode = 0444; - repo_git_path_replace(the_repository, temp_filename, "objects/%s", pattern); - fd = git_mkstemp_mode(temp_filename->buf, mode); - if (0 <= fd) - return fd; - - /* slow path */ - /* some mkstemp implementations erase temp_filename on failure */ - repo_git_path_replace(the_repository, temp_filename, "objects/%s", pattern); - safe_create_leading_directories(temp_filename->buf); - return xmkstemp_mode(temp_filename->buf, mode); -} - -int odb_pack_keep(const char *name) -{ - int fd; - - fd = open(name, O_RDWR|O_CREAT|O_EXCL, 0600); - if (0 <= fd) - return fd; - - /* slow path */ - safe_create_leading_directories_const(name); - return open(name, O_RDWR|O_CREAT|O_EXCL, 0600); -} - static void fill_loose_path(struct strbuf *buf, const struct object_id *oid) { int i; @@ -137,9 +54,9 @@ static void fill_loose_path(struct strbuf *buf, const struct object_id *oid) } } -static const char *odb_loose_path(struct object_directory *odb, - struct strbuf *buf, - const struct object_id *oid) +const char *odb_loose_path(struct object_directory *odb, + struct strbuf *buf, + const struct object_id *oid) { strbuf_reset(buf); strbuf_addstr(buf, odb->path); @@ -148,513 +65,6 @@ static const char *odb_loose_path(struct object_directory *odb, return buf->buf; } -const char *loose_object_path(struct repository *r, struct strbuf *buf, - const struct object_id *oid) -{ - return odb_loose_path(r->objects->odb, buf, oid); -} - -/* - * Return non-zero iff the path is usable as an alternate object database. - */ -static int alt_odb_usable(struct raw_object_store *o, - struct strbuf *path, - const char *normalized_objdir, khiter_t *pos) -{ - int r; - - /* Detect cases where alternate disappeared */ - if (!is_directory(path->buf)) { - error(_("object directory %s does not exist; " - "check .git/objects/info/alternates"), - path->buf); - return 0; - } - - /* - * Prevent the common mistake of listing the same - * thing twice, or object directory itself. - */ - if (!o->odb_by_path) { - khiter_t p; - - o->odb_by_path = kh_init_odb_path_map(); - assert(!o->odb->next); - p = kh_put_odb_path_map(o->odb_by_path, o->odb->path, &r); - assert(r == 1); /* never used */ - kh_value(o->odb_by_path, p) = o->odb; - } - if (fspatheq(path->buf, normalized_objdir)) - return 0; - *pos = kh_put_odb_path_map(o->odb_by_path, path->buf, &r); - /* r: 0 = exists, 1 = never used, 2 = deleted */ - return r == 0 ? 0 : 1; -} - -/* - * Prepare alternate object database registry. - * - * The variable alt_odb_list points at the list of struct - * object_directory. The elements on this list come from - * non-empty elements from colon separated ALTERNATE_DB_ENVIRONMENT - * environment variable, and $GIT_OBJECT_DIRECTORY/info/alternates, - * whose contents is similar to that environment variable but can be - * LF separated. Its base points at a statically allocated buffer that - * contains "/the/directory/corresponding/to/.git/objects/...", while - * its name points just after the slash at the end of ".git/objects/" - * in the example above, and has enough space to hold all hex characters - * of the object ID, an extra slash for the first level indirection, and - * the terminating NUL. - */ -static void read_info_alternates(struct repository *r, - const char *relative_base, - int depth); -static int link_alt_odb_entry(struct repository *r, const struct strbuf *entry, - const char *relative_base, int depth, const char *normalized_objdir) -{ - struct object_directory *ent; - struct strbuf pathbuf = STRBUF_INIT; - struct strbuf tmp = STRBUF_INIT; - khiter_t pos; - int ret = -1; - - if (!is_absolute_path(entry->buf) && relative_base) { - strbuf_realpath(&pathbuf, relative_base, 1); - strbuf_addch(&pathbuf, '/'); - } - strbuf_addbuf(&pathbuf, entry); - - if (!strbuf_realpath(&tmp, pathbuf.buf, 0)) { - error(_("unable to normalize alternate object path: %s"), - pathbuf.buf); - goto error; - } - strbuf_swap(&pathbuf, &tmp); - - /* - * The trailing slash after the directory name is given by - * this function at the end. Remove duplicates. - */ - while (pathbuf.len && pathbuf.buf[pathbuf.len - 1] == '/') - strbuf_setlen(&pathbuf, pathbuf.len - 1); - - if (!alt_odb_usable(r->objects, &pathbuf, normalized_objdir, &pos)) - goto error; - - CALLOC_ARRAY(ent, 1); - /* pathbuf.buf is already in r->objects->odb_by_path */ - ent->path = strbuf_detach(&pathbuf, NULL); - - /* add the alternate entry */ - *r->objects->odb_tail = ent; - r->objects->odb_tail = &(ent->next); - ent->next = NULL; - assert(r->objects->odb_by_path); - kh_value(r->objects->odb_by_path, pos) = ent; - - /* recursively add alternates */ - read_info_alternates(r, ent->path, depth + 1); - ret = 0; - error: - strbuf_release(&tmp); - strbuf_release(&pathbuf); - return ret; -} - -static const char *parse_alt_odb_entry(const char *string, - int sep, - struct strbuf *out) -{ - const char *end; - - strbuf_reset(out); - - if (*string == '#') { - /* comment; consume up to next separator */ - end = strchrnul(string, sep); - } else if (*string == '"' && !unquote_c_style(out, string, &end)) { - /* - * quoted path; unquote_c_style has copied the - * data for us and set "end". Broken quoting (e.g., - * an entry that doesn't end with a quote) falls - * back to the unquoted case below. - */ - } else { - /* normal, unquoted path */ - end = strchrnul(string, sep); - strbuf_add(out, string, end - string); - } - - if (*end) - end++; - return end; -} - -static void link_alt_odb_entries(struct repository *r, const char *alt, - int sep, const char *relative_base, int depth) -{ - struct strbuf objdirbuf = STRBUF_INIT; - struct strbuf entry = STRBUF_INIT; - - if (!alt || !*alt) - return; - - if (depth > 5) { - error(_("%s: ignoring alternate object stores, nesting too deep"), - relative_base); - return; - } - - strbuf_realpath(&objdirbuf, r->objects->odb->path, 1); - - while (*alt) { - alt = parse_alt_odb_entry(alt, sep, &entry); - if (!entry.len) - continue; - link_alt_odb_entry(r, &entry, - relative_base, depth, objdirbuf.buf); - } - strbuf_release(&entry); - strbuf_release(&objdirbuf); -} - -static void read_info_alternates(struct repository *r, - const char *relative_base, - int depth) -{ - char *path; - struct strbuf buf = STRBUF_INIT; - - path = xstrfmt("%s/info/alternates", relative_base); - if (strbuf_read_file(&buf, path, 1024) < 0) { - warn_on_fopen_errors(path); - free(path); - return; - } - - link_alt_odb_entries(r, buf.buf, '\n', relative_base, depth); - strbuf_release(&buf); - free(path); -} - -void add_to_alternates_file(const char *reference) -{ - struct lock_file lock = LOCK_INIT; - char *alts = repo_git_path(the_repository, "objects/info/alternates"); - FILE *in, *out; - int found = 0; - - hold_lock_file_for_update(&lock, alts, LOCK_DIE_ON_ERROR); - out = fdopen_lock_file(&lock, "w"); - if (!out) - die_errno(_("unable to fdopen alternates lockfile")); - - in = fopen(alts, "r"); - if (in) { - struct strbuf line = STRBUF_INIT; - - while (strbuf_getline(&line, in) != EOF) { - if (!strcmp(reference, line.buf)) { - found = 1; - break; - } - fprintf_or_die(out, "%s\n", line.buf); - } - - strbuf_release(&line); - fclose(in); - } - else if (errno != ENOENT) - die_errno(_("unable to read alternates file")); - - if (found) { - rollback_lock_file(&lock); - } else { - fprintf_or_die(out, "%s\n", reference); - if (commit_lock_file(&lock)) - die_errno(_("unable to move new alternates file into place")); - if (the_repository->objects->loaded_alternates) - link_alt_odb_entries(the_repository, reference, - '\n', NULL, 0); - } - free(alts); -} - -void add_to_alternates_memory(const char *reference) -{ - /* - * Make sure alternates are initialized, or else our entry may be - * overwritten when they are. - */ - prepare_alt_odb(the_repository); - - link_alt_odb_entries(the_repository, reference, - '\n', NULL, 0); -} - -struct object_directory *set_temporary_primary_odb(const char *dir, int will_destroy) -{ - struct object_directory *new_odb; - - /* - * Make sure alternates are initialized, or else our entry may be - * overwritten when they are. - */ - prepare_alt_odb(the_repository); - - /* - * Make a new primary odb and link the old primary ODB in as an - * alternate - */ - new_odb = xcalloc(1, sizeof(*new_odb)); - new_odb->path = xstrdup(dir); - - /* - * Disable ref updates while a temporary odb is active, since - * the objects in the database may roll back. - */ - new_odb->disable_ref_updates = 1; - new_odb->will_destroy = will_destroy; - new_odb->next = the_repository->objects->odb; - the_repository->objects->odb = new_odb; - return new_odb->next; -} - -void restore_primary_odb(struct object_directory *restore_odb, const char *old_path) -{ - struct object_directory *cur_odb = the_repository->objects->odb; - - if (strcmp(old_path, cur_odb->path)) - BUG("expected %s as primary object store; found %s", - old_path, cur_odb->path); - - if (cur_odb->next != restore_odb) - BUG("we expect the old primary object store to be the first alternate"); - - the_repository->objects->odb = restore_odb; - free_object_directory(cur_odb); -} - -/* - * Compute the exact path an alternate is at and returns it. In case of - * error NULL is returned and the human readable error is added to `err` - * `path` may be relative and should point to $GIT_DIR. - * `err` must not be null. - */ -char *compute_alternate_path(const char *path, struct strbuf *err) -{ - char *ref_git = NULL; - const char *repo; - int seen_error = 0; - - ref_git = real_pathdup(path, 0); - if (!ref_git) { - seen_error = 1; - strbuf_addf(err, _("path '%s' does not exist"), path); - goto out; - } - - repo = read_gitfile(ref_git); - if (!repo) - repo = read_gitfile(mkpath("%s/.git", ref_git)); - if (repo) { - free(ref_git); - ref_git = xstrdup(repo); - } - - if (!repo && is_directory(mkpath("%s/.git/objects", ref_git))) { - char *ref_git_git = mkpathdup("%s/.git", ref_git); - free(ref_git); - ref_git = ref_git_git; - } else if (!is_directory(mkpath("%s/objects", ref_git))) { - struct strbuf sb = STRBUF_INIT; - seen_error = 1; - if (get_common_dir(&sb, ref_git)) { - strbuf_addf(err, - _("reference repository '%s' as a linked " - "checkout is not supported yet."), - path); - goto out; - } - - strbuf_addf(err, _("reference repository '%s' is not a " - "local repository."), path); - goto out; - } - - if (!access(mkpath("%s/shallow", ref_git), F_OK)) { - strbuf_addf(err, _("reference repository '%s' is shallow"), - path); - seen_error = 1; - goto out; - } - - if (!access(mkpath("%s/info/grafts", ref_git), F_OK)) { - strbuf_addf(err, - _("reference repository '%s' is grafted"), - path); - seen_error = 1; - goto out; - } - -out: - if (seen_error) { - FREE_AND_NULL(ref_git); - } - - return ref_git; -} - -struct object_directory *find_odb(struct repository *r, const char *obj_dir) -{ - struct object_directory *odb; - char *obj_dir_real = real_pathdup(obj_dir, 1); - struct strbuf odb_path_real = STRBUF_INIT; - - prepare_alt_odb(r); - for (odb = r->objects->odb; odb; odb = odb->next) { - strbuf_realpath(&odb_path_real, odb->path, 1); - if (!strcmp(obj_dir_real, odb_path_real.buf)) - break; - } - - free(obj_dir_real); - strbuf_release(&odb_path_real); - - if (!odb) - die(_("could not find object directory matching %s"), obj_dir); - return odb; -} - -static void fill_alternate_refs_command(struct child_process *cmd, - const char *repo_path) -{ - const char *value; - - if (!git_config_get_value("core.alternateRefsCommand", &value)) { - cmd->use_shell = 1; - - strvec_push(&cmd->args, value); - strvec_push(&cmd->args, repo_path); - } else { - cmd->git_cmd = 1; - - strvec_pushf(&cmd->args, "--git-dir=%s", repo_path); - strvec_push(&cmd->args, "for-each-ref"); - strvec_push(&cmd->args, "--format=%(objectname)"); - - if (!git_config_get_value("core.alternateRefsPrefixes", &value)) { - strvec_push(&cmd->args, "--"); - strvec_split(&cmd->args, value); - } - } - - strvec_pushv(&cmd->env, (const char **)local_repo_env); - cmd->out = -1; -} - -static void read_alternate_refs(const char *path, - alternate_ref_fn *cb, - void *data) -{ - struct child_process cmd = CHILD_PROCESS_INIT; - struct strbuf line = STRBUF_INIT; - FILE *fh; - - fill_alternate_refs_command(&cmd, path); - - if (start_command(&cmd)) - return; - - fh = xfdopen(cmd.out, "r"); - while (strbuf_getline_lf(&line, fh) != EOF) { - struct object_id oid; - const char *p; - - if (parse_oid_hex(line.buf, &oid, &p) || *p) { - warning(_("invalid line while parsing alternate refs: %s"), - line.buf); - break; - } - - cb(&oid, data); - } - - fclose(fh); - finish_command(&cmd); - strbuf_release(&line); -} - -struct alternate_refs_data { - alternate_ref_fn *fn; - void *data; -}; - -static int refs_from_alternate_cb(struct object_directory *e, - void *data) -{ - struct strbuf path = STRBUF_INIT; - size_t base_len; - struct alternate_refs_data *cb = data; - - if (!strbuf_realpath(&path, e->path, 0)) - goto out; - if (!strbuf_strip_suffix(&path, "/objects")) - goto out; - base_len = path.len; - - /* Is this a git repository with refs? */ - strbuf_addstr(&path, "/refs"); - if (!is_directory(path.buf)) - goto out; - strbuf_setlen(&path, base_len); - - read_alternate_refs(path.buf, cb->fn, cb->data); - -out: - strbuf_release(&path); - return 0; -} - -void for_each_alternate_ref(alternate_ref_fn fn, void *data) -{ - struct alternate_refs_data cb; - cb.fn = fn; - cb.data = data; - foreach_alt_odb(refs_from_alternate_cb, &cb); -} - -int foreach_alt_odb(alt_odb_fn fn, void *cb) -{ - struct object_directory *ent; - int r = 0; - - prepare_alt_odb(the_repository); - for (ent = the_repository->objects->odb->next; ent; ent = ent->next) { - r = fn(ent, cb); - if (r) - break; - } - return r; -} - -void prepare_alt_odb(struct repository *r) -{ - if (r->objects->loaded_alternates) - return; - - link_alt_odb_entries(r, r->objects->alternate_db, PATH_SEP, NULL, 0); - - read_info_alternates(r, r->objects->odb->path, 0); - r->objects->loaded_alternates = 1; -} - -int has_alt_odb(struct repository *r) -{ - prepare_alt_odb(r); - return !!r->objects->odb->next; -} - /* Returns 1 if we have successfully freshened the file, 0 otherwise. */ static int freshen_file(const char *fn) { @@ -1056,9 +466,9 @@ int parse_loose_header(const char *hdr, struct object_info *oi) return 0; } -static int loose_object_info(struct repository *r, - const struct object_id *oid, - struct object_info *oi, int flags) +int loose_object_info(struct repository *r, + const struct object_id *oid, + struct object_info *oi, int flags) { int status = 0; int fd; @@ -1154,345 +564,6 @@ static int loose_object_info(struct repository *r, return status; } -int obj_read_use_lock = 0; -pthread_mutex_t obj_read_mutex; - -void enable_obj_read_lock(void) -{ - if (obj_read_use_lock) - return; - - obj_read_use_lock = 1; - init_recursive_mutex(&obj_read_mutex); -} - -void disable_obj_read_lock(void) -{ - if (!obj_read_use_lock) - return; - - obj_read_use_lock = 0; - pthread_mutex_destroy(&obj_read_mutex); -} - -int fetch_if_missing = 1; - -static int do_oid_object_info_extended(struct repository *r, - const struct object_id *oid, - struct object_info *oi, unsigned flags) -{ - static struct object_info blank_oi = OBJECT_INFO_INIT; - const struct cached_object *co; - struct pack_entry e; - int rtype; - const struct object_id *real = oid; - int already_retried = 0; - - - if (flags & OBJECT_INFO_LOOKUP_REPLACE) - real = lookup_replace_object(r, oid); - - if (is_null_oid(real)) - return -1; - - if (!oi) - oi = &blank_oi; - - co = find_cached_object(real); - if (co) { - if (oi->typep) - *(oi->typep) = co->type; - if (oi->sizep) - *(oi->sizep) = co->size; - if (oi->disk_sizep) - *(oi->disk_sizep) = 0; - if (oi->delta_base_oid) - oidclr(oi->delta_base_oid, the_repository->hash_algo); - if (oi->type_name) - strbuf_addstr(oi->type_name, type_name(co->type)); - if (oi->contentp) - *oi->contentp = xmemdupz(co->buf, co->size); - oi->whence = OI_CACHED; - return 0; - } - - while (1) { - if (find_pack_entry(r, real, &e)) - break; - - /* Most likely it's a loose object. */ - if (!loose_object_info(r, real, oi, flags)) - return 0; - - /* Not a loose object; someone else may have just packed it. */ - if (!(flags & OBJECT_INFO_QUICK)) { - reprepare_packed_git(r); - if (find_pack_entry(r, real, &e)) - break; - } - - /* - * If r is the_repository, this might be an attempt at - * accessing a submodule object as if it were in the_repository - * (having called add_submodule_odb() on that submodule's ODB). - * If any such ODBs exist, register them and try again. - */ - if (r == the_repository && - register_all_submodule_odb_as_alternates()) - /* We added some alternates; retry */ - continue; - - /* Check if it is a missing object */ - if (fetch_if_missing && repo_has_promisor_remote(r) && - !already_retried && - !(flags & OBJECT_INFO_SKIP_FETCH_OBJECT)) { - promisor_remote_get_direct(r, real, 1); - already_retried = 1; - continue; - } - - if (flags & OBJECT_INFO_DIE_IF_CORRUPT) { - const struct packed_git *p; - if ((flags & OBJECT_INFO_LOOKUP_REPLACE) && !oideq(real, oid)) - die(_("replacement %s not found for %s"), - oid_to_hex(real), oid_to_hex(oid)); - if ((p = has_packed_and_bad(r, real))) - die(_("packed object %s (stored in %s) is corrupt"), - oid_to_hex(real), p->pack_name); - } - return -1; - } - - if (oi == &blank_oi) - /* - * We know that the caller doesn't actually need the - * information below, so return early. - */ - return 0; - rtype = packed_object_info(r, e.p, e.offset, oi); - if (rtype < 0) { - mark_bad_packed_object(e.p, real); - return do_oid_object_info_extended(r, real, oi, 0); - } else if (oi->whence == OI_PACKED) { - oi->u.packed.offset = e.offset; - oi->u.packed.pack = e.p; - oi->u.packed.is_delta = (rtype == OBJ_REF_DELTA || - rtype == OBJ_OFS_DELTA); - } - - return 0; -} - -static int oid_object_info_convert(struct repository *r, - const struct object_id *input_oid, - struct object_info *input_oi, unsigned flags) -{ - const struct git_hash_algo *input_algo = &hash_algos[input_oid->algo]; - int do_die = flags & OBJECT_INFO_DIE_IF_CORRUPT; - struct strbuf type_name = STRBUF_INIT; - struct object_id oid, delta_base_oid; - struct object_info new_oi, *oi; - unsigned long size; - void *content; - int ret; - - if (repo_oid_to_algop(r, input_oid, the_hash_algo, &oid)) { - if (do_die) - die(_("missing mapping of %s to %s"), - oid_to_hex(input_oid), the_hash_algo->name); - return -1; - } - - /* Is new_oi needed? */ - oi = input_oi; - if (input_oi && (input_oi->delta_base_oid || input_oi->sizep || - input_oi->contentp)) { - new_oi = *input_oi; - /* Does delta_base_oid need to be converted? */ - if (input_oi->delta_base_oid) - new_oi.delta_base_oid = &delta_base_oid; - /* Will the attributes differ when converted? */ - if (input_oi->sizep || input_oi->contentp) { - new_oi.contentp = &content; - new_oi.sizep = &size; - new_oi.type_name = &type_name; - } - oi = &new_oi; - } - - ret = oid_object_info_extended(r, &oid, oi, flags); - if (ret) - return -1; - if (oi == input_oi) - return ret; - - if (new_oi.contentp) { - struct strbuf outbuf = STRBUF_INIT; - enum object_type type; - - type = type_from_string_gently(type_name.buf, type_name.len, - !do_die); - if (type == -1) - return -1; - if (type != OBJ_BLOB) { - ret = convert_object_file(the_repository, &outbuf, - the_hash_algo, input_algo, - content, size, type, !do_die); - free(content); - if (ret == -1) - return -1; - size = outbuf.len; - content = strbuf_detach(&outbuf, NULL); - } - if (input_oi->sizep) - *input_oi->sizep = size; - if (input_oi->contentp) - *input_oi->contentp = content; - else - free(content); - if (input_oi->type_name) - *input_oi->type_name = type_name; - else - strbuf_release(&type_name); - } - if (new_oi.delta_base_oid == &delta_base_oid) { - if (repo_oid_to_algop(r, &delta_base_oid, input_algo, - input_oi->delta_base_oid)) { - if (do_die) - die(_("missing mapping of %s to %s"), - oid_to_hex(&delta_base_oid), - input_algo->name); - return -1; - } - } - input_oi->whence = new_oi.whence; - input_oi->u = new_oi.u; - return ret; -} - -int oid_object_info_extended(struct repository *r, const struct object_id *oid, - struct object_info *oi, unsigned flags) -{ - int ret; - - if (oid->algo && (hash_algo_by_ptr(r->hash_algo) != oid->algo)) - return oid_object_info_convert(r, oid, oi, flags); - - obj_read_lock(); - ret = do_oid_object_info_extended(r, oid, oi, flags); - obj_read_unlock(); - return ret; -} - - -/* returns enum object_type or negative */ -int oid_object_info(struct repository *r, - const struct object_id *oid, - unsigned long *sizep) -{ - enum object_type type; - struct object_info oi = OBJECT_INFO_INIT; - - oi.typep = &type; - oi.sizep = sizep; - if (oid_object_info_extended(r, oid, &oi, - OBJECT_INFO_LOOKUP_REPLACE) < 0) - return -1; - return type; -} - -int pretend_object_file(void *buf, unsigned long len, enum object_type type, - struct object_id *oid) -{ - struct cached_object_entry *co; - char *co_buf; - - hash_object_file(the_hash_algo, buf, len, type, oid); - if (repo_has_object_file_with_flags(the_repository, oid, OBJECT_INFO_QUICK | OBJECT_INFO_SKIP_FETCH_OBJECT) || - find_cached_object(oid)) - return 0; - ALLOC_GROW(cached_objects, cached_object_nr + 1, cached_object_alloc); - co = &cached_objects[cached_object_nr++]; - co->value.size = len; - co->value.type = type; - co_buf = xmalloc(len); - memcpy(co_buf, buf, len); - co->value.buf = co_buf; - oidcpy(&co->oid, oid); - return 0; -} - -/* - * This function dies on corrupt objects; the callers who want to - * deal with them should arrange to call oid_object_info_extended() and give - * error messages themselves. - */ -void *repo_read_object_file(struct repository *r, - const struct object_id *oid, - enum object_type *type, - unsigned long *size) -{ - struct object_info oi = OBJECT_INFO_INIT; - unsigned flags = OBJECT_INFO_DIE_IF_CORRUPT | OBJECT_INFO_LOOKUP_REPLACE; - void *data; - - oi.typep = type; - oi.sizep = size; - oi.contentp = &data; - if (oid_object_info_extended(r, oid, &oi, flags)) - return NULL; - - return data; -} - -void *read_object_with_reference(struct repository *r, - const struct object_id *oid, - enum object_type required_type, - unsigned long *size, - struct object_id *actual_oid_return) -{ - enum object_type type; - void *buffer; - unsigned long isize; - struct object_id actual_oid; - - oidcpy(&actual_oid, oid); - while (1) { - int ref_length = -1; - const char *ref_type = NULL; - - buffer = repo_read_object_file(r, &actual_oid, &type, &isize); - if (!buffer) - return NULL; - if (type == required_type) { - *size = isize; - if (actual_oid_return) - oidcpy(actual_oid_return, &actual_oid); - return buffer; - } - /* Handle references */ - else if (type == OBJ_COMMIT) - ref_type = "tree "; - else if (type == OBJ_TAG) - ref_type = "object "; - else { - free(buffer); - return NULL; - } - ref_length = strlen(ref_type); - - if (ref_length + the_hash_algo->hexsz > isize || - memcmp(buffer, ref_type, ref_length) || - get_oid_hex((char *) buffer + ref_length, &actual_oid)) { - free(buffer); - return NULL; - } - free(buffer); - /* Now we have the ID of the referred-to object in - * actual_oid. Check again. */ - } -} - static void hash_object_body(const struct git_hash_algo *algo, struct git_hash_ctx *c, const void *buf, unsigned long len, struct object_id *oid, @@ -2154,32 +1225,6 @@ int force_object_loose(const struct object_id *oid, time_t mtime) return ret; } -int has_object(struct repository *r, const struct object_id *oid, - unsigned flags) -{ - int quick = !(flags & HAS_OBJECT_RECHECK_PACKED); - unsigned object_info_flags = OBJECT_INFO_SKIP_FETCH_OBJECT | - (quick ? OBJECT_INFO_QUICK : 0); - - if (!startup_info->have_repository) - return 0; - return oid_object_info_extended(r, oid, NULL, object_info_flags) >= 0; -} - -int repo_has_object_file_with_flags(struct repository *r, - const struct object_id *oid, int flags) -{ - if (!startup_info->have_repository) - return 0; - return oid_object_info_extended(r, oid, NULL, flags) >= 0; -} - -int repo_has_object_file(struct repository *r, - const struct object_id *oid) -{ - return repo_has_object_file_with_flags(r, oid, 0); -} - /* * We can't use the normal fsck_error_function() for index_mem(), * because we don't yet have a valid oid for it to report. Instead, @@ -2407,16 +1452,6 @@ int read_pack_header(int fd, struct pack_header *header) return 0; } -void assert_oid_type(const struct object_id *oid, enum object_type expect) -{ - enum object_type type = oid_object_info(the_repository, oid, NULL); - if (type < 0) - die(_("%s is not a valid object"), oid_to_hex(oid)); - if (type != expect) - die(_("%s is not a valid '%s' object"), oid_to_hex(oid), - type_name(expect)); -} - int for_each_file_in_obj_subdir(unsigned int subdir_nr, struct strbuf *path, each_loose_object_fn obj_cb, diff --git a/object-file.h b/object-file.h index 353d8a85c33..78c84d970a9 100644 --- a/object-file.h +++ b/object-file.h @@ -21,6 +21,29 @@ extern int fetch_if_missing; int index_fd(struct index_state *istate, struct object_id *oid, int fd, struct stat *st, enum object_type type, const char *path, unsigned flags); int index_path(struct index_state *istate, struct object_id *oid, const char *path, struct stat *st, unsigned flags); +struct object_directory; + +const char *odb_loose_path(struct object_directory *odb, + struct strbuf *buf, + const struct object_id *oid); + +/* + * Return true iff an alternate object database has a loose object + * with the specified name. This function does not respect replace + * references. + */ +int has_loose_object_nonlocal(const struct object_id *); + +int has_loose_object(const struct object_id *); + +/** + * format_object_header() is a thin wrapper around s xsnprintf() that + * writes the initial " " part of the loose object + * header. It returns the size that snprintf() returns + 1. + */ +int format_object_header(char *str, size_t size, enum object_type type, + size_t objsize); + /** * unpack_loose_header() initializes the data stream needed to unpack * a loose object header. @@ -61,6 +84,29 @@ enum unpack_loose_header_result unpack_loose_header(git_zstream *stream, struct object_info; int parse_loose_header(const char *hdr, struct object_info *oi); +int write_object_file_flags(const void *buf, unsigned long len, + enum object_type type, struct object_id *oid, + struct object_id *comapt_oid_in, unsigned flags); +static inline int write_object_file(const void *buf, unsigned long len, + enum object_type type, struct object_id *oid) +{ + return write_object_file_flags(buf, len, type, oid, NULL, 0); +} + +struct input_stream { + const void *(*read)(struct input_stream *, unsigned long *len); + void *data; + int is_finished; +}; + +int write_object_file_literally(const void *buf, unsigned long len, + const char *type, struct object_id *oid, + unsigned flags); +int stream_loose_object(struct input_stream *in_stream, size_t len, + struct object_id *oid); + +int force_object_loose(const struct object_id *oid, time_t mtime); + /** * With in-core object data in "buf", rehash it to make sure the * object name actually matches "oid" to detect object corruption. @@ -79,6 +125,10 @@ int check_object_signature(struct repository *r, const struct object_id *oid, */ int stream_object_signature(struct repository *r, const struct object_id *oid); +int loose_object_info(struct repository *r, + const struct object_id *oid, + struct object_info *oi, int flags); + enum finalize_object_file_flags { FOF_SKIP_COLLISION_CHECK = 1, }; @@ -90,10 +140,18 @@ int finalize_object_file_flags(const char *tmpfile, const char *filename, /* Helper to check and "touch" a file */ int check_and_freshen_file(const char *fn, int freshen); -void *read_object_with_reference(struct repository *r, - const struct object_id *oid, - enum object_type required_type, - unsigned long *size, - struct object_id *oid_ret); +/* + * Open the loose object at path, check its hash, and return the contents, + * use the "oi" argument to assert things about the object, or e.g. populate its + * type, and size. If the object is a blob, then "contents" may return NULL, + * to allow streaming of large blobs. + * + * Returns 0 on success, negative on error (details may be written to stderr). + */ +int read_loose_object(const char *path, + const struct object_id *expected_oid, + struct object_id *real_oid, + void **contents, + struct object_info *oi); #endif /* OBJECT_FILE_H */ diff --git a/object-store-ll.h b/object-store-ll.h index cd3bd5bd99f..8ae80b8a5fa 100644 --- a/object-store-ll.h +++ b/object-store-ll.h @@ -49,12 +49,6 @@ struct object_directory { char *path; }; -struct input_stream { - const void *(*read)(struct input_stream *, unsigned long *len); - void *data; - int is_finished; -}; - void prepare_alt_odb(struct repository *r); int has_alt_odb(struct repository *r); char *compute_alternate_path(const char *path, struct strbuf *err); @@ -273,21 +267,6 @@ void hash_object_file(const struct git_hash_algo *algo, const void *buf, unsigned long len, enum object_type type, struct object_id *oid); -int write_object_file_flags(const void *buf, unsigned long len, - enum object_type type, struct object_id *oid, - struct object_id *comapt_oid_in, unsigned flags); -static inline int write_object_file(const void *buf, unsigned long len, - enum object_type type, struct object_id *oid) -{ - return write_object_file_flags(buf, len, type, oid, NULL, 0); -} - -int write_object_file_literally(const void *buf, unsigned long len, - const char *type, struct object_id *oid, - unsigned flags); -int stream_loose_object(struct input_stream *in_stream, size_t len, - struct object_id *oid); - /* * Add an object file to the in-memory object store, without writing it * to disk. @@ -299,8 +278,6 @@ int stream_loose_object(struct input_stream *in_stream, size_t len, int pretend_object_file(void *, unsigned long, enum object_type, struct object_id *oid); -int force_object_loose(const struct object_id *oid, time_t mtime); - struct object_info { /* Request */ enum object_type *typep; @@ -364,20 +341,6 @@ int oid_object_info_extended(struct repository *r, const struct object_id *, struct object_info *, unsigned flags); -/* - * Open the loose object at path, check its hash, and return the contents, - * use the "oi" argument to assert things about the object, or e.g. populate its - * type, and size. If the object is a blob, then "contents" may return NULL, - * to allow streaming of large blobs. - * - * Returns 0 on success, negative on error (details may be written to stderr). - */ -int read_loose_object(const char *path, - const struct object_id *expected_oid, - struct object_id *real_oid, - void **contents, - struct object_info *oi); - /* Retry packed storage after checking packed and loose storage */ #define HAS_OBJECT_RECHECK_PACKED 1 @@ -405,23 +368,6 @@ int repo_has_object_file(struct repository *r, const struct object_id *oid); int repo_has_object_file_with_flags(struct repository *r, const struct object_id *oid, int flags); -/* - * Return true iff an alternate object database has a loose object - * with the specified name. This function does not respect replace - * references. - */ -int has_loose_object_nonlocal(const struct object_id *); - -int has_loose_object(const struct object_id *); - -/** - * format_object_header() is a thin wrapper around s xsnprintf() that - * writes the initial " " part of the loose object - * header. It returns the size that snprintf() returns + 1. - */ -int format_object_header(char *str, size_t size, enum object_type type, - size_t objsize); - void assert_oid_type(const struct object_id *oid, enum object_type expect); /* @@ -553,4 +499,10 @@ int for_each_object_in_pack(struct packed_git *p, int for_each_packed_object(struct repository *repo, each_packed_object_fn cb, void *data, enum for_each_object_flags flags); +void *read_object_with_reference(struct repository *r, + const struct object_id *oid, + enum object_type required_type, + unsigned long *size, + struct object_id *oid_ret); + #endif /* OBJECT_STORE_LL_H */ diff --git a/object-store.c b/object-store.c new file mode 100644 index 00000000000..e5f1f00cdde --- /dev/null +++ b/object-store.c @@ -0,0 +1,972 @@ +#define USE_THE_REPOSITORY_VARIABLE + +#include "git-compat-util.h" +#include "abspath.h" +#include "config.h" +#include "environment.h" +#include "gettext.h" +#include "hex.h" +#include "lockfile.h" +#include "object-file-convert.h" +#include "object-file.h" +#include "object-store.h" +#include "packfile.h" +#include "path.h" +#include "promisor-remote.h" +#include "quote.h" +#include "replace-object.h" +#include "run-command.h" +#include "setup.h" +#include "strbuf.h" +#include "strvec.h" +#include "submodule.h" +#include "write-or-die.h" + +/* + * This is meant to hold a *small* number of objects that you would + * want repo_read_object_file() to be able to return, but yet you do not want + * to write them into the object store (e.g. a browse-only + * application). + */ +static struct cached_object_entry { + struct object_id oid; + struct cached_object { + enum object_type type; + const void *buf; + unsigned long size; + } value; +} *cached_objects; +static int cached_object_nr, cached_object_alloc; + +static const struct cached_object *find_cached_object(const struct object_id *oid) +{ + static const struct cached_object empty_tree = { + .type = OBJ_TREE, + .buf = "", + }; + int i; + const struct cached_object_entry *co = cached_objects; + + for (i = 0; i < cached_object_nr; i++, co++) { + if (oideq(&co->oid, oid)) + return &co->value; + } + if (oideq(oid, the_hash_algo->empty_tree)) + return &empty_tree; + return NULL; +} + +int odb_mkstemp(struct strbuf *temp_filename, const char *pattern) +{ + int fd; + /* + * we let the umask do its job, don't try to be more + * restrictive except to remove write permission. + */ + int mode = 0444; + repo_git_path_replace(the_repository, temp_filename, "objects/%s", pattern); + fd = git_mkstemp_mode(temp_filename->buf, mode); + if (0 <= fd) + return fd; + + /* slow path */ + /* some mkstemp implementations erase temp_filename on failure */ + repo_git_path_replace(the_repository, temp_filename, "objects/%s", pattern); + safe_create_leading_directories(temp_filename->buf); + return xmkstemp_mode(temp_filename->buf, mode); +} + +int odb_pack_keep(const char *name) +{ + int fd; + + fd = open(name, O_RDWR|O_CREAT|O_EXCL, 0600); + if (0 <= fd) + return fd; + + /* slow path */ + safe_create_leading_directories_const(name); + return open(name, O_RDWR|O_CREAT|O_EXCL, 0600); +} + +const char *loose_object_path(struct repository *r, struct strbuf *buf, + const struct object_id *oid) +{ + return odb_loose_path(r->objects->odb, buf, oid); +} + +/* + * Return non-zero iff the path is usable as an alternate object database. + */ +static int alt_odb_usable(struct raw_object_store *o, + struct strbuf *path, + const char *normalized_objdir, khiter_t *pos) +{ + int r; + + /* Detect cases where alternate disappeared */ + if (!is_directory(path->buf)) { + error(_("object directory %s does not exist; " + "check .git/objects/info/alternates"), + path->buf); + return 0; + } + + /* + * Prevent the common mistake of listing the same + * thing twice, or object directory itself. + */ + if (!o->odb_by_path) { + khiter_t p; + + o->odb_by_path = kh_init_odb_path_map(); + assert(!o->odb->next); + p = kh_put_odb_path_map(o->odb_by_path, o->odb->path, &r); + assert(r == 1); /* never used */ + kh_value(o->odb_by_path, p) = o->odb; + } + if (fspatheq(path->buf, normalized_objdir)) + return 0; + *pos = kh_put_odb_path_map(o->odb_by_path, path->buf, &r); + /* r: 0 = exists, 1 = never used, 2 = deleted */ + return r == 0 ? 0 : 1; +} + +/* + * Prepare alternate object database registry. + * + * The variable alt_odb_list points at the list of struct + * object_directory. The elements on this list come from + * non-empty elements from colon separated ALTERNATE_DB_ENVIRONMENT + * environment variable, and $GIT_OBJECT_DIRECTORY/info/alternates, + * whose contents is similar to that environment variable but can be + * LF separated. Its base points at a statically allocated buffer that + * contains "/the/directory/corresponding/to/.git/objects/...", while + * its name points just after the slash at the end of ".git/objects/" + * in the example above, and has enough space to hold all hex characters + * of the object ID, an extra slash for the first level indirection, and + * the terminating NUL. + */ +static void read_info_alternates(struct repository *r, + const char *relative_base, + int depth); +static int link_alt_odb_entry(struct repository *r, const struct strbuf *entry, + const char *relative_base, int depth, const char *normalized_objdir) +{ + struct object_directory *ent; + struct strbuf pathbuf = STRBUF_INIT; + struct strbuf tmp = STRBUF_INIT; + khiter_t pos; + int ret = -1; + + if (!is_absolute_path(entry->buf) && relative_base) { + strbuf_realpath(&pathbuf, relative_base, 1); + strbuf_addch(&pathbuf, '/'); + } + strbuf_addbuf(&pathbuf, entry); + + if (!strbuf_realpath(&tmp, pathbuf.buf, 0)) { + error(_("unable to normalize alternate object path: %s"), + pathbuf.buf); + goto error; + } + strbuf_swap(&pathbuf, &tmp); + + /* + * The trailing slash after the directory name is given by + * this function at the end. Remove duplicates. + */ + while (pathbuf.len && pathbuf.buf[pathbuf.len - 1] == '/') + strbuf_setlen(&pathbuf, pathbuf.len - 1); + + if (!alt_odb_usable(r->objects, &pathbuf, normalized_objdir, &pos)) + goto error; + + CALLOC_ARRAY(ent, 1); + /* pathbuf.buf is already in r->objects->odb_by_path */ + ent->path = strbuf_detach(&pathbuf, NULL); + + /* add the alternate entry */ + *r->objects->odb_tail = ent; + r->objects->odb_tail = &(ent->next); + ent->next = NULL; + assert(r->objects->odb_by_path); + kh_value(r->objects->odb_by_path, pos) = ent; + + /* recursively add alternates */ + read_info_alternates(r, ent->path, depth + 1); + ret = 0; + error: + strbuf_release(&tmp); + strbuf_release(&pathbuf); + return ret; +} + +static const char *parse_alt_odb_entry(const char *string, + int sep, + struct strbuf *out) +{ + const char *end; + + strbuf_reset(out); + + if (*string == '#') { + /* comment; consume up to next separator */ + end = strchrnul(string, sep); + } else if (*string == '"' && !unquote_c_style(out, string, &end)) { + /* + * quoted path; unquote_c_style has copied the + * data for us and set "end". Broken quoting (e.g., + * an entry that doesn't end with a quote) falls + * back to the unquoted case below. + */ + } else { + /* normal, unquoted path */ + end = strchrnul(string, sep); + strbuf_add(out, string, end - string); + } + + if (*end) + end++; + return end; +} + +static void link_alt_odb_entries(struct repository *r, const char *alt, + int sep, const char *relative_base, int depth) +{ + struct strbuf objdirbuf = STRBUF_INIT; + struct strbuf entry = STRBUF_INIT; + + if (!alt || !*alt) + return; + + if (depth > 5) { + error(_("%s: ignoring alternate object stores, nesting too deep"), + relative_base); + return; + } + + strbuf_realpath(&objdirbuf, r->objects->odb->path, 1); + + while (*alt) { + alt = parse_alt_odb_entry(alt, sep, &entry); + if (!entry.len) + continue; + link_alt_odb_entry(r, &entry, + relative_base, depth, objdirbuf.buf); + } + strbuf_release(&entry); + strbuf_release(&objdirbuf); +} + +static void read_info_alternates(struct repository *r, + const char *relative_base, + int depth) +{ + char *path; + struct strbuf buf = STRBUF_INIT; + + path = xstrfmt("%s/info/alternates", relative_base); + if (strbuf_read_file(&buf, path, 1024) < 0) { + warn_on_fopen_errors(path); + free(path); + return; + } + + link_alt_odb_entries(r, buf.buf, '\n', relative_base, depth); + strbuf_release(&buf); + free(path); +} + +void add_to_alternates_file(const char *reference) +{ + struct lock_file lock = LOCK_INIT; + char *alts = repo_git_path(the_repository, "objects/info/alternates"); + FILE *in, *out; + int found = 0; + + hold_lock_file_for_update(&lock, alts, LOCK_DIE_ON_ERROR); + out = fdopen_lock_file(&lock, "w"); + if (!out) + die_errno(_("unable to fdopen alternates lockfile")); + + in = fopen(alts, "r"); + if (in) { + struct strbuf line = STRBUF_INIT; + + while (strbuf_getline(&line, in) != EOF) { + if (!strcmp(reference, line.buf)) { + found = 1; + break; + } + fprintf_or_die(out, "%s\n", line.buf); + } + + strbuf_release(&line); + fclose(in); + } + else if (errno != ENOENT) + die_errno(_("unable to read alternates file")); + + if (found) { + rollback_lock_file(&lock); + } else { + fprintf_or_die(out, "%s\n", reference); + if (commit_lock_file(&lock)) + die_errno(_("unable to move new alternates file into place")); + if (the_repository->objects->loaded_alternates) + link_alt_odb_entries(the_repository, reference, + '\n', NULL, 0); + } + free(alts); +} + +void add_to_alternates_memory(const char *reference) +{ + /* + * Make sure alternates are initialized, or else our entry may be + * overwritten when they are. + */ + prepare_alt_odb(the_repository); + + link_alt_odb_entries(the_repository, reference, + '\n', NULL, 0); +} + +struct object_directory *set_temporary_primary_odb(const char *dir, int will_destroy) +{ + struct object_directory *new_odb; + + /* + * Make sure alternates are initialized, or else our entry may be + * overwritten when they are. + */ + prepare_alt_odb(the_repository); + + /* + * Make a new primary odb and link the old primary ODB in as an + * alternate + */ + new_odb = xcalloc(1, sizeof(*new_odb)); + new_odb->path = xstrdup(dir); + + /* + * Disable ref updates while a temporary odb is active, since + * the objects in the database may roll back. + */ + new_odb->disable_ref_updates = 1; + new_odb->will_destroy = will_destroy; + new_odb->next = the_repository->objects->odb; + the_repository->objects->odb = new_odb; + return new_odb->next; +} + +void restore_primary_odb(struct object_directory *restore_odb, const char *old_path) +{ + struct object_directory *cur_odb = the_repository->objects->odb; + + if (strcmp(old_path, cur_odb->path)) + BUG("expected %s as primary object store; found %s", + old_path, cur_odb->path); + + if (cur_odb->next != restore_odb) + BUG("we expect the old primary object store to be the first alternate"); + + the_repository->objects->odb = restore_odb; + free_object_directory(cur_odb); +} + +/* + * Compute the exact path an alternate is at and returns it. In case of + * error NULL is returned and the human readable error is added to `err` + * `path` may be relative and should point to $GIT_DIR. + * `err` must not be null. + */ +char *compute_alternate_path(const char *path, struct strbuf *err) +{ + char *ref_git = NULL; + const char *repo; + int seen_error = 0; + + ref_git = real_pathdup(path, 0); + if (!ref_git) { + seen_error = 1; + strbuf_addf(err, _("path '%s' does not exist"), path); + goto out; + } + + repo = read_gitfile(ref_git); + if (!repo) + repo = read_gitfile(mkpath("%s/.git", ref_git)); + if (repo) { + free(ref_git); + ref_git = xstrdup(repo); + } + + if (!repo && is_directory(mkpath("%s/.git/objects", ref_git))) { + char *ref_git_git = mkpathdup("%s/.git", ref_git); + free(ref_git); + ref_git = ref_git_git; + } else if (!is_directory(mkpath("%s/objects", ref_git))) { + struct strbuf sb = STRBUF_INIT; + seen_error = 1; + if (get_common_dir(&sb, ref_git)) { + strbuf_addf(err, + _("reference repository '%s' as a linked " + "checkout is not supported yet."), + path); + goto out; + } + + strbuf_addf(err, _("reference repository '%s' is not a " + "local repository."), path); + goto out; + } + + if (!access(mkpath("%s/shallow", ref_git), F_OK)) { + strbuf_addf(err, _("reference repository '%s' is shallow"), + path); + seen_error = 1; + goto out; + } + + if (!access(mkpath("%s/info/grafts", ref_git), F_OK)) { + strbuf_addf(err, + _("reference repository '%s' is grafted"), + path); + seen_error = 1; + goto out; + } + +out: + if (seen_error) { + FREE_AND_NULL(ref_git); + } + + return ref_git; +} + +struct object_directory *find_odb(struct repository *r, const char *obj_dir) +{ + struct object_directory *odb; + char *obj_dir_real = real_pathdup(obj_dir, 1); + struct strbuf odb_path_real = STRBUF_INIT; + + prepare_alt_odb(r); + for (odb = r->objects->odb; odb; odb = odb->next) { + strbuf_realpath(&odb_path_real, odb->path, 1); + if (!strcmp(obj_dir_real, odb_path_real.buf)) + break; + } + + free(obj_dir_real); + strbuf_release(&odb_path_real); + + if (!odb) + die(_("could not find object directory matching %s"), obj_dir); + return odb; +} + +static void fill_alternate_refs_command(struct child_process *cmd, + const char *repo_path) +{ + const char *value; + + if (!git_config_get_value("core.alternateRefsCommand", &value)) { + cmd->use_shell = 1; + + strvec_push(&cmd->args, value); + strvec_push(&cmd->args, repo_path); + } else { + cmd->git_cmd = 1; + + strvec_pushf(&cmd->args, "--git-dir=%s", repo_path); + strvec_push(&cmd->args, "for-each-ref"); + strvec_push(&cmd->args, "--format=%(objectname)"); + + if (!git_config_get_value("core.alternateRefsPrefixes", &value)) { + strvec_push(&cmd->args, "--"); + strvec_split(&cmd->args, value); + } + } + + strvec_pushv(&cmd->env, (const char **)local_repo_env); + cmd->out = -1; +} + +static void read_alternate_refs(const char *path, + alternate_ref_fn *cb, + void *data) +{ + struct child_process cmd = CHILD_PROCESS_INIT; + struct strbuf line = STRBUF_INIT; + FILE *fh; + + fill_alternate_refs_command(&cmd, path); + + if (start_command(&cmd)) + return; + + fh = xfdopen(cmd.out, "r"); + while (strbuf_getline_lf(&line, fh) != EOF) { + struct object_id oid; + const char *p; + + if (parse_oid_hex(line.buf, &oid, &p) || *p) { + warning(_("invalid line while parsing alternate refs: %s"), + line.buf); + break; + } + + cb(&oid, data); + } + + fclose(fh); + finish_command(&cmd); + strbuf_release(&line); +} + +struct alternate_refs_data { + alternate_ref_fn *fn; + void *data; +}; + +static int refs_from_alternate_cb(struct object_directory *e, + void *data) +{ + struct strbuf path = STRBUF_INIT; + size_t base_len; + struct alternate_refs_data *cb = data; + + if (!strbuf_realpath(&path, e->path, 0)) + goto out; + if (!strbuf_strip_suffix(&path, "/objects")) + goto out; + base_len = path.len; + + /* Is this a git repository with refs? */ + strbuf_addstr(&path, "/refs"); + if (!is_directory(path.buf)) + goto out; + strbuf_setlen(&path, base_len); + + read_alternate_refs(path.buf, cb->fn, cb->data); + +out: + strbuf_release(&path); + return 0; +} + +void for_each_alternate_ref(alternate_ref_fn fn, void *data) +{ + struct alternate_refs_data cb; + cb.fn = fn; + cb.data = data; + foreach_alt_odb(refs_from_alternate_cb, &cb); +} + +int foreach_alt_odb(alt_odb_fn fn, void *cb) +{ + struct object_directory *ent; + int r = 0; + + prepare_alt_odb(the_repository); + for (ent = the_repository->objects->odb->next; ent; ent = ent->next) { + r = fn(ent, cb); + if (r) + break; + } + return r; +} + +void prepare_alt_odb(struct repository *r) +{ + if (r->objects->loaded_alternates) + return; + + link_alt_odb_entries(r, r->objects->alternate_db, PATH_SEP, NULL, 0); + + read_info_alternates(r, r->objects->odb->path, 0); + r->objects->loaded_alternates = 1; +} + +int has_alt_odb(struct repository *r) +{ + prepare_alt_odb(r); + return !!r->objects->odb->next; +} + +int obj_read_use_lock = 0; +pthread_mutex_t obj_read_mutex; + +void enable_obj_read_lock(void) +{ + if (obj_read_use_lock) + return; + + obj_read_use_lock = 1; + init_recursive_mutex(&obj_read_mutex); +} + +void disable_obj_read_lock(void) +{ + if (!obj_read_use_lock) + return; + + obj_read_use_lock = 0; + pthread_mutex_destroy(&obj_read_mutex); +} + +int fetch_if_missing = 1; + +static int do_oid_object_info_extended(struct repository *r, + const struct object_id *oid, + struct object_info *oi, unsigned flags) +{ + static struct object_info blank_oi = OBJECT_INFO_INIT; + const struct cached_object *co; + struct pack_entry e; + int rtype; + const struct object_id *real = oid; + int already_retried = 0; + + + if (flags & OBJECT_INFO_LOOKUP_REPLACE) + real = lookup_replace_object(r, oid); + + if (is_null_oid(real)) + return -1; + + if (!oi) + oi = &blank_oi; + + co = find_cached_object(real); + if (co) { + if (oi->typep) + *(oi->typep) = co->type; + if (oi->sizep) + *(oi->sizep) = co->size; + if (oi->disk_sizep) + *(oi->disk_sizep) = 0; + if (oi->delta_base_oid) + oidclr(oi->delta_base_oid, the_repository->hash_algo); + if (oi->type_name) + strbuf_addstr(oi->type_name, type_name(co->type)); + if (oi->contentp) + *oi->contentp = xmemdupz(co->buf, co->size); + oi->whence = OI_CACHED; + return 0; + } + + while (1) { + if (find_pack_entry(r, real, &e)) + break; + + /* Most likely it's a loose object. */ + if (!loose_object_info(r, real, oi, flags)) + return 0; + + /* Not a loose object; someone else may have just packed it. */ + if (!(flags & OBJECT_INFO_QUICK)) { + reprepare_packed_git(r); + if (find_pack_entry(r, real, &e)) + break; + } + + /* + * If r is the_repository, this might be an attempt at + * accessing a submodule object as if it were in the_repository + * (having called add_submodule_odb() on that submodule's ODB). + * If any such ODBs exist, register them and try again. + */ + if (r == the_repository && + register_all_submodule_odb_as_alternates()) + /* We added some alternates; retry */ + continue; + + /* Check if it is a missing object */ + if (fetch_if_missing && repo_has_promisor_remote(r) && + !already_retried && + !(flags & OBJECT_INFO_SKIP_FETCH_OBJECT)) { + promisor_remote_get_direct(r, real, 1); + already_retried = 1; + continue; + } + + if (flags & OBJECT_INFO_DIE_IF_CORRUPT) { + const struct packed_git *p; + if ((flags & OBJECT_INFO_LOOKUP_REPLACE) && !oideq(real, oid)) + die(_("replacement %s not found for %s"), + oid_to_hex(real), oid_to_hex(oid)); + if ((p = has_packed_and_bad(r, real))) + die(_("packed object %s (stored in %s) is corrupt"), + oid_to_hex(real), p->pack_name); + } + return -1; + } + + if (oi == &blank_oi) + /* + * We know that the caller doesn't actually need the + * information below, so return early. + */ + return 0; + rtype = packed_object_info(r, e.p, e.offset, oi); + if (rtype < 0) { + mark_bad_packed_object(e.p, real); + return do_oid_object_info_extended(r, real, oi, 0); + } else if (oi->whence == OI_PACKED) { + oi->u.packed.offset = e.offset; + oi->u.packed.pack = e.p; + oi->u.packed.is_delta = (rtype == OBJ_REF_DELTA || + rtype == OBJ_OFS_DELTA); + } + + return 0; +} + +static int oid_object_info_convert(struct repository *r, + const struct object_id *input_oid, + struct object_info *input_oi, unsigned flags) +{ + const struct git_hash_algo *input_algo = &hash_algos[input_oid->algo]; + int do_die = flags & OBJECT_INFO_DIE_IF_CORRUPT; + struct strbuf type_name = STRBUF_INIT; + struct object_id oid, delta_base_oid; + struct object_info new_oi, *oi; + unsigned long size; + void *content; + int ret; + + if (repo_oid_to_algop(r, input_oid, the_hash_algo, &oid)) { + if (do_die) + die(_("missing mapping of %s to %s"), + oid_to_hex(input_oid), the_hash_algo->name); + return -1; + } + + /* Is new_oi needed? */ + oi = input_oi; + if (input_oi && (input_oi->delta_base_oid || input_oi->sizep || + input_oi->contentp)) { + new_oi = *input_oi; + /* Does delta_base_oid need to be converted? */ + if (input_oi->delta_base_oid) + new_oi.delta_base_oid = &delta_base_oid; + /* Will the attributes differ when converted? */ + if (input_oi->sizep || input_oi->contentp) { + new_oi.contentp = &content; + new_oi.sizep = &size; + new_oi.type_name = &type_name; + } + oi = &new_oi; + } + + ret = oid_object_info_extended(r, &oid, oi, flags); + if (ret) + return -1; + if (oi == input_oi) + return ret; + + if (new_oi.contentp) { + struct strbuf outbuf = STRBUF_INIT; + enum object_type type; + + type = type_from_string_gently(type_name.buf, type_name.len, + !do_die); + if (type == -1) + return -1; + if (type != OBJ_BLOB) { + ret = convert_object_file(the_repository, &outbuf, + the_hash_algo, input_algo, + content, size, type, !do_die); + free(content); + if (ret == -1) + return -1; + size = outbuf.len; + content = strbuf_detach(&outbuf, NULL); + } + if (input_oi->sizep) + *input_oi->sizep = size; + if (input_oi->contentp) + *input_oi->contentp = content; + else + free(content); + if (input_oi->type_name) + *input_oi->type_name = type_name; + else + strbuf_release(&type_name); + } + if (new_oi.delta_base_oid == &delta_base_oid) { + if (repo_oid_to_algop(r, &delta_base_oid, input_algo, + input_oi->delta_base_oid)) { + if (do_die) + die(_("missing mapping of %s to %s"), + oid_to_hex(&delta_base_oid), + input_algo->name); + return -1; + } + } + input_oi->whence = new_oi.whence; + input_oi->u = new_oi.u; + return ret; +} + +int oid_object_info_extended(struct repository *r, const struct object_id *oid, + struct object_info *oi, unsigned flags) +{ + int ret; + + if (oid->algo && (hash_algo_by_ptr(r->hash_algo) != oid->algo)) + return oid_object_info_convert(r, oid, oi, flags); + + obj_read_lock(); + ret = do_oid_object_info_extended(r, oid, oi, flags); + obj_read_unlock(); + return ret; +} + + +/* returns enum object_type or negative */ +int oid_object_info(struct repository *r, + const struct object_id *oid, + unsigned long *sizep) +{ + enum object_type type; + struct object_info oi = OBJECT_INFO_INIT; + + oi.typep = &type; + oi.sizep = sizep; + if (oid_object_info_extended(r, oid, &oi, + OBJECT_INFO_LOOKUP_REPLACE) < 0) + return -1; + return type; +} + +int pretend_object_file(void *buf, unsigned long len, enum object_type type, + struct object_id *oid) +{ + struct cached_object_entry *co; + char *co_buf; + + hash_object_file(the_hash_algo, buf, len, type, oid); + if (repo_has_object_file_with_flags(the_repository, oid, OBJECT_INFO_QUICK | OBJECT_INFO_SKIP_FETCH_OBJECT) || + find_cached_object(oid)) + return 0; + ALLOC_GROW(cached_objects, cached_object_nr + 1, cached_object_alloc); + co = &cached_objects[cached_object_nr++]; + co->value.size = len; + co->value.type = type; + co_buf = xmalloc(len); + memcpy(co_buf, buf, len); + co->value.buf = co_buf; + oidcpy(&co->oid, oid); + return 0; +} + +/* + * This function dies on corrupt objects; the callers who want to + * deal with them should arrange to call oid_object_info_extended() and give + * error messages themselves. + */ +void *repo_read_object_file(struct repository *r, + const struct object_id *oid, + enum object_type *type, + unsigned long *size) +{ + struct object_info oi = OBJECT_INFO_INIT; + unsigned flags = OBJECT_INFO_DIE_IF_CORRUPT | OBJECT_INFO_LOOKUP_REPLACE; + void *data; + + oi.typep = type; + oi.sizep = size; + oi.contentp = &data; + if (oid_object_info_extended(r, oid, &oi, flags)) + return NULL; + + return data; +} + +void *read_object_with_reference(struct repository *r, + const struct object_id *oid, + enum object_type required_type, + unsigned long *size, + struct object_id *actual_oid_return) +{ + enum object_type type; + void *buffer; + unsigned long isize; + struct object_id actual_oid; + + oidcpy(&actual_oid, oid); + while (1) { + int ref_length = -1; + const char *ref_type = NULL; + + buffer = repo_read_object_file(r, &actual_oid, &type, &isize); + if (!buffer) + return NULL; + if (type == required_type) { + *size = isize; + if (actual_oid_return) + oidcpy(actual_oid_return, &actual_oid); + return buffer; + } + /* Handle references */ + else if (type == OBJ_COMMIT) + ref_type = "tree "; + else if (type == OBJ_TAG) + ref_type = "object "; + else { + free(buffer); + return NULL; + } + ref_length = strlen(ref_type); + + if (ref_length + the_hash_algo->hexsz > isize || + memcmp(buffer, ref_type, ref_length) || + get_oid_hex((char *) buffer + ref_length, &actual_oid)) { + free(buffer); + return NULL; + } + free(buffer); + /* Now we have the ID of the referred-to object in + * actual_oid. Check again. */ + } +} + +int has_object(struct repository *r, const struct object_id *oid, + unsigned flags) +{ + int quick = !(flags & HAS_OBJECT_RECHECK_PACKED); + unsigned object_info_flags = OBJECT_INFO_SKIP_FETCH_OBJECT | + (quick ? OBJECT_INFO_QUICK : 0); + + if (!startup_info->have_repository) + return 0; + return oid_object_info_extended(r, oid, NULL, object_info_flags) >= 0; +} + +int repo_has_object_file_with_flags(struct repository *r, + const struct object_id *oid, int flags) +{ + if (!startup_info->have_repository) + return 0; + return oid_object_info_extended(r, oid, NULL, flags) >= 0; +} + +int repo_has_object_file(struct repository *r, + const struct object_id *oid) +{ + return repo_has_object_file_with_flags(r, oid, 0); +} + +void assert_oid_type(const struct object_id *oid, enum object_type expect) +{ + enum object_type type = oid_object_info(the_repository, oid, NULL); + if (type < 0) + die(_("%s is not a valid object"), oid_to_hex(oid)); + if (type != expect) + die(_("%s is not a valid '%s' object"), oid_to_hex(oid), + type_name(expect)); +} From patchwork Tue Apr 8 10:24:13 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Patrick Steinhardt X-Patchwork-Id: 14042666 Received: from fout-a6-smtp.messagingengine.com (fout-a6-smtp.messagingengine.com [103.168.172.149]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 811EB2690F8 for ; Tue, 8 Apr 2025 10:24:19 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=103.168.172.149 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1744107861; cv=none; b=qWej+HUs6FL3MUNv87P0UutKiSa+8ScFXyLwV+ORPRg1+oBvg8GTWMS3T9Job0J/U5ou//DDQ3BOSjyLsDJ6PaSXUVl+KGShJ76St7nQLFMb0n03A2kp6RilNiIojaIbKpws63DlBL/Z+g5L9MPegFecPqHr1UmAsGOE1k3UQho= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1744107861; c=relaxed/simple; bh=kptPemjWN/fguUigEmQ2XrXy1O3bPDuNEpUYAbqv84U=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=OLBl8lEfdHXACNr3rtgSuJjVL2onLpZlLIdao2rCmpjHyTe6ga12kyda3CIwKLAYof6XNcY/LOReeA6tkOmeiPQNXq/DWp/8wsR5OtI1hGjWw/by9qzQD8AAdQkGJRsGIKP/Y0uIW5AbRi1xVK4Rv3wTHGjMlYag5M9eDSkmczU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=pks.im; spf=pass smtp.mailfrom=pks.im; dkim=pass (2048-bit key) header.d=pks.im header.i=@pks.im header.b=eZ1bSLc/; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b=LqLl0aeI; arc=none smtp.client-ip=103.168.172.149 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=pks.im Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=pks.im Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=pks.im header.i=@pks.im header.b="eZ1bSLc/"; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b="LqLl0aeI" Received: from phl-compute-09.internal (phl-compute-09.phl.internal [10.202.2.49]) by mailfout.phl.internal (Postfix) with ESMTP id 733AD138019E for ; Tue, 8 Apr 2025 06:24:18 -0400 (EDT) Received: from phl-mailfrontend-02 ([10.202.2.163]) by phl-compute-09.internal (MEProxy); Tue, 08 Apr 2025 06:24:18 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=pks.im; h=cc:cc :content-transfer-encoding:content-type:content-type:date:date :from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:subject:subject:to:to; s=fm2; t=1744107858; x=1744194258; bh=7lImVkEo0V7vFBE+m74sKBTZAk1o811bK/PCNLwEGMQ=; b= eZ1bSLc/KtiniFohXefF7HlBYJh8Qe7utFU9voNQwS28nn6X+x1RqVkn1xv6fNpo F54jtTBG5QsW1tAOQBjq59/7QZely0WOQXTmWRCBe1MLubkSBsRDAQz53B7GQdoQ C7wghgnr0qcDRqVja26uF42mlINwjjdz34GInJ24RZECubwDRIg21eDBUa0Kre0g OMYCnlrAMy71dO475re06aiYOnJQqD7YoeFlZy2CeAg+B2okonS7g2vF/XKVMHeu 0GGA3R1b+CGQCNBqq47iK34usPxofgVvmyu5B2r+od11hpCWOfZRJtOwOsMxryzn 0t/mdKhBjmMIjOqGkFAnlQ== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-transfer-encoding :content-type:content-type:date:date:feedback-id:feedback-id :from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:subject:subject:to:to:x-me-proxy :x-me-sender:x-me-sender:x-sasl-enc; s=fm2; t=1744107858; x= 1744194258; bh=7lImVkEo0V7vFBE+m74sKBTZAk1o811bK/PCNLwEGMQ=; b=L qLl0aeI5UjYfgTg9fLteozDhdw0uLmYurZc9XTX3fbTejST0K215BVRPnpKmFOBi obW4Cwc4+EAGr0M+aD5LkPUGaabaAearV66APs9B1b6jiPLA7J7Gg9Q82W9/2bJN R9w+qabpit3XBwIObzzt6ASjYbv2cwI4TlA+HO2v4egzggdwmTNM0oCqHVKqzaAn ogFE/EknZk6B/sqjfDTaBr90HPCaF51/ptVljpJ5NL8A1ZZ67uHUU7NFhxgDxJvb sGMtI8V9cT4FVHCgHc4Iou6e3M+IV0HYZ0AciWqDbCgrsO2KeIsJW1RiybehkiPw 4knnk/+7Qye+ywpzOGkcA== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeefvddrtddtgddvtddvkeegucetufdoteggodetrf dotffvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdggtfgfnhhsuhgsshgtrhhisggv pdfurfetoffkrfgpnffqhgenuceurghilhhouhhtmecufedttdenucenucfjughrpefhff fugggtgffkfhgjvfevofesthejredtredtjeenucfhrhhomheprfgrthhrihgtkhcuufht vghinhhhrghrughtuceophhssehpkhhsrdhimheqnecuggftrfgrthhtvghrnhepffeuie dujedvkeehuedvkeefffeivdeuleetkeduheejteekgedvudfgtdfgieelnecuvehluhhs thgvrhfuihiivgeptdenucfrrghrrghmpehmrghilhhfrhhomhepphhssehpkhhsrdhimh dpnhgspghrtghpthhtohepuddpmhhouggvpehsmhhtphhouhhtpdhrtghpthhtohepghhi thesvhhgvghrrdhkvghrnhgvlhdrohhrgh X-ME-Proxy: Feedback-ID: i197146af:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA for ; Tue, 8 Apr 2025 06:24:17 -0400 (EDT) Received: by vm-mail (OpenSMTPD) with ESMTPSA id d9c2612d (TLSv1.3:TLS_AES_256_GCM_SHA384:256:NO) for ; Tue, 8 Apr 2025 10:24:16 +0000 (UTC) From: Patrick Steinhardt Date: Tue, 08 Apr 2025 12:24:13 +0200 Subject: [PATCH 5/9] object-file: split up concerns of `HASH_*` flags Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20250408-pks-split-object-file-v1-5-f1fd50191143@pks.im> References: <20250408-pks-split-object-file-v1-0-f1fd50191143@pks.im> In-Reply-To: <20250408-pks-split-object-file-v1-0-f1fd50191143@pks.im> To: git@vger.kernel.org Cc: X-Mailer: b4 0.14.2 The functions `hash_object_file()`, `write_object_file()` and `index_fd()` reuse the same set of flags to alter their behaviour. This not only adds confusion, but given that every function only supports a subset of the flags it becomes very hard to see which flags can be passed to what function. Last but not least, this entangles the implementation of all three function families. Split up concerns by creating separate flags for each of the function families. Signed-off-by: Patrick Steinhardt --- builtin/hash-object.c | 23 +++++++++++++++++------ builtin/replace.c | 2 +- builtin/update-index.c | 2 +- bulk-checkin.c | 6 +++--- cache-tree.c | 2 +- notes-merge.c | 2 +- object-file.c | 18 +++++++++--------- object-file.h | 25 +++++++++++++++++++++---- read-cache.c | 4 ++-- 9 files changed, 56 insertions(+), 28 deletions(-) diff --git a/builtin/hash-object.c b/builtin/hash-object.c index a25f0403f44..e7c0d6afdef 100644 --- a/builtin/hash-object.c +++ b/builtin/hash-object.c @@ -19,6 +19,11 @@ #include "strbuf.h" #include "write-or-die.h" +enum { + HASH_OBJECT_CHECK = (1 << 0), + HASH_OBJECT_WRITE = (1 << 1), +}; + /* * This is to create corrupt objects for debugging and as such it * needs to bypass the data conversion performed by, and the type @@ -33,7 +38,7 @@ static int hash_literally(struct object_id *oid, int fd, const char *type, unsig ret = -1; else ret = write_object_file_literally(buf.buf, buf.len, type, oid, - flags); + (flags & HASH_OBJECT_WRITE) ? WRITE_OBJECT_FILE_PERSIST : 0); close(fd); strbuf_release(&buf); return ret; @@ -42,15 +47,21 @@ static int hash_literally(struct object_id *oid, int fd, const char *type, unsig static void hash_fd(int fd, const char *type, const char *path, unsigned flags, int literally) { + unsigned int index_flags = 0; struct stat st; struct object_id oid; + if (flags & HASH_OBJECT_WRITE) + index_flags |= INDEX_WRITE_OBJECT; + if (flags & HASH_OBJECT_CHECK) + index_flags |= INDEX_FORMAT_CHECK; + if (fstat(fd, &st) < 0 || (literally ? hash_literally(&oid, fd, type, flags) : index_fd(the_repository->index, &oid, fd, &st, - type_from_string(type), path, flags))) - die((flags & HASH_WRITE_OBJECT) + type_from_string(type), path, index_flags))) + die((flags & HASH_OBJECT_WRITE) ? "Unable to add %s to database" : "Unable to hash %s", path); printf("%s\n", oid_to_hex(&oid)); @@ -102,13 +113,13 @@ int cmd_hash_object(int argc, int no_filters = 0; int literally = 0; int nongit = 0; - unsigned flags = HASH_FORMAT_CHECK; + unsigned flags = HASH_OBJECT_CHECK; const char *vpath = NULL; char *vpath_free = NULL; const struct option hash_object_options[] = { OPT_STRING('t', NULL, &type, N_("type"), N_("object type")), OPT_BIT('w', NULL, &flags, N_("write the object into the object database"), - HASH_WRITE_OBJECT), + HASH_OBJECT_WRITE), OPT_COUNTUP( 0 , "stdin", &hashstdin, N_("read the object from stdin")), OPT_BOOL( 0 , "stdin-paths", &stdin_paths, N_("read file names from stdin")), OPT_BOOL( 0 , "no-filters", &no_filters, N_("store file as is without filters")), @@ -122,7 +133,7 @@ int cmd_hash_object(int argc, argc = parse_options(argc, argv, prefix, hash_object_options, hash_object_usage, 0); - if (flags & HASH_WRITE_OBJECT) + if (flags & HASH_OBJECT_WRITE) prefix = setup_git_directory(); else prefix = setup_git_directory_gently(&nongit); diff --git a/builtin/replace.c b/builtin/replace.c index 15ec0922ce1..2b4fc9a68b3 100644 --- a/builtin/replace.c +++ b/builtin/replace.c @@ -305,7 +305,7 @@ static int import_object(struct object_id *oid, enum object_type type, strbuf_release(&result); } else { struct stat st; - int flags = HASH_FORMAT_CHECK | HASH_WRITE_OBJECT; + int flags = INDEX_FORMAT_CHECK | INDEX_WRITE_OBJECT; if (fstat(fd, &st) < 0) { error_errno(_("unable to fstat %s"), filename); diff --git a/builtin/update-index.c b/builtin/update-index.c index b2f6b1a3fbb..f0cf964294d 100644 --- a/builtin/update-index.c +++ b/builtin/update-index.c @@ -304,7 +304,7 @@ static int add_one_path(const struct cache_entry *old, const char *path, int len ce->ce_mode = ce_mode_from_stat(old, st->st_mode); if (index_path(the_repository->index, &ce->oid, path, st, - info_only ? 0 : HASH_WRITE_OBJECT)) { + info_only ? 0 : INDEX_WRITE_OBJECT)) { discard_cache_entry(ce); return -1; } diff --git a/bulk-checkin.c b/bulk-checkin.c index 23ac00ea0a6..309201a76a6 100644 --- a/bulk-checkin.c +++ b/bulk-checkin.c @@ -171,7 +171,7 @@ static int stream_blob_to_pack(struct bulk_checkin_packfile *state, unsigned char obuf[16384]; unsigned hdrlen; int status = Z_OK; - int write_object = (flags & HASH_WRITE_OBJECT); + int write_object = (flags & INDEX_WRITE_OBJECT); off_t offset = 0; git_deflate_init(&s, pack_compression_level); @@ -241,7 +241,7 @@ static int stream_blob_to_pack(struct bulk_checkin_packfile *state, static void prepare_to_stream(struct bulk_checkin_packfile *state, unsigned flags) { - if (!(flags & HASH_WRITE_OBJECT) || state->f) + if (!(flags & INDEX_WRITE_OBJECT) || state->f) return; state->f = create_tmp_packfile(the_repository, &state->pack_tmp_name); @@ -275,7 +275,7 @@ static int deflate_blob_to_pack(struct bulk_checkin_packfile *state, git_hash_update(&ctx, obuf, header_len); /* Note: idx is non-NULL when we are writing */ - if ((flags & HASH_WRITE_OBJECT) != 0) { + if ((flags & INDEX_WRITE_OBJECT) != 0) { CALLOC_ARRAY(idx, 1); prepare_to_stream(state, flags); diff --git a/cache-tree.c b/cache-tree.c index bcbcad3d61a..4c8167ea927 100644 --- a/cache-tree.c +++ b/cache-tree.c @@ -452,7 +452,7 @@ static int update_one(struct cache_tree *it, OBJ_TREE, &it->oid); } else if (write_object_file_flags(buffer.buf, buffer.len, OBJ_TREE, &it->oid, NULL, flags & WRITE_TREE_SILENT - ? HASH_SILENT : 0)) { + ? WRITE_OBJECT_FILE_SILENT : 0)) { strbuf_release(&buffer); return -1; } diff --git a/notes-merge.c b/notes-merge.c index 5008faef450..d9f2914c819 100644 --- a/notes-merge.c +++ b/notes-merge.c @@ -729,7 +729,7 @@ int notes_merge_commit(struct notes_merge_options *o, /* write file as blob, and add to partial_tree */ if (stat(path.buf, &st)) die_errno("Failed to stat '%s'", path.buf); - if (index_path(o->repo->index, &blob_oid, path.buf, &st, HASH_WRITE_OBJECT)) + if (index_path(o->repo->index, &blob_oid, path.buf, &st, INDEX_WRITE_OBJECT)) die("Failed to write blob object from '%s'", path.buf); if (add_note(partial_tree, &obj_oid, &blob_oid, NULL)) die("Failed to add resolved note '%s' to notes tree", diff --git a/object-file.c b/object-file.c index 1c74aa239fb..dea5a70a331 100644 --- a/object-file.c +++ b/object-file.c @@ -33,9 +33,9 @@ static int get_conv_flags(unsigned flags) { - if (flags & HASH_RENORMALIZE) + if (flags & INDEX_RENORMALIZE) return CONV_EOL_RENORMALIZE; - else if (flags & HASH_WRITE_OBJECT) + else if (flags & INDEX_WRITE_OBJECT) return global_conv_flags_eol | CONV_WRITE_OBJECT; else return 0; @@ -835,7 +835,7 @@ static int start_loose_object_common(struct strbuf *tmp_file, fd = create_tmpfile(tmp_file, filename); if (fd < 0) { - if (flags & HASH_SILENT) + if (flags & WRITE_OBJECT_FILE_SILENT) return -1; else if (errno == EACCES) return error(_("insufficient permission for adding " @@ -967,7 +967,7 @@ static int write_loose_object(const struct object_id *oid, char *hdr, utb.actime = mtime; utb.modtime = mtime; if (utime(tmp_file.buf, &utb) < 0 && - !(flags & HASH_SILENT)) + !(flags & WRITE_OBJECT_FILE_SILENT)) warning_errno(_("failed utime() on %s"), tmp_file.buf); } @@ -1178,7 +1178,7 @@ int write_object_file_literally(const void *buf, unsigned long len, write_object_file_prepare_literally(the_hash_algo, buf, len, type, oid, header, &hdrlen); - if (!(flags & HASH_WRITE_OBJECT)) + if (!(flags & WRITE_OBJECT_FILE_PERSIST)) goto cleanup; if (freshen_packed_object(oid) || freshen_loose_object(oid)) goto cleanup; @@ -1249,7 +1249,7 @@ static int index_mem(struct index_state *istate, { struct strbuf nbuf = STRBUF_INIT; int ret = 0; - int write_object = flags & HASH_WRITE_OBJECT; + int write_object = flags & INDEX_WRITE_OBJECT; if (!type) type = OBJ_BLOB; @@ -1264,7 +1264,7 @@ static int index_mem(struct index_state *istate, size = nbuf.len; } } - if (flags & HASH_FORMAT_CHECK) { + if (flags & INDEX_FORMAT_CHECK) { struct fsck_options opts = FSCK_OPTIONS_DEFAULT; opts.strict = 1; @@ -1290,7 +1290,7 @@ static int index_stream_convert_blob(struct index_state *istate, unsigned flags) { int ret = 0; - const int write_object = flags & HASH_WRITE_OBJECT; + const int write_object = flags & INDEX_WRITE_OBJECT; struct strbuf sbuf = STRBUF_INIT; assert(path); @@ -1422,7 +1422,7 @@ int index_path(struct index_state *istate, struct object_id *oid, case S_IFLNK: if (strbuf_readlink(&sb, path, st->st_size)) return error_errno("readlink(\"%s\")", path); - if (!(flags & HASH_WRITE_OBJECT)) + if (!(flags & INDEX_WRITE_OBJECT)) hash_object_file(the_hash_algo, sb.buf, sb.len, OBJ_BLOB, oid); else if (write_object_file(sb.buf, sb.len, OBJ_BLOB, oid)) diff --git a/object-file.h b/object-file.h index 78c84d970a9..c002fbe2345 100644 --- a/object-file.h +++ b/object-file.h @@ -14,10 +14,12 @@ struct index_state; */ extern int fetch_if_missing; -#define HASH_WRITE_OBJECT 1 -#define HASH_FORMAT_CHECK 2 -#define HASH_RENORMALIZE 4 -#define HASH_SILENT 8 +enum { + INDEX_WRITE_OBJECT = (1 << 0), + INDEX_FORMAT_CHECK = (1 << 1), + INDEX_RENORMALIZE = (1 << 2), +}; + int index_fd(struct index_state *istate, struct object_id *oid, int fd, struct stat *st, enum object_type type, const char *path, unsigned flags); int index_path(struct index_state *istate, struct object_id *oid, const char *path, struct stat *st, unsigned flags); @@ -84,6 +86,21 @@ enum unpack_loose_header_result unpack_loose_header(git_zstream *stream, struct object_info; int parse_loose_header(const char *hdr, struct object_info *oi); +enum { + /* + * By default, `write_object_file_literally()` does not actually write + * anything into the object store, but only computes the object ID. + * This flag changes that so that the object will be written as a loose + * object and persisted. + */ + WRITE_OBJECT_FILE_PERSIST = (1 << 0), + + /* + * Do not print an error in case something gose wrong. + */ + WRITE_OBJECT_FILE_SILENT = (1 << 1), +}; + int write_object_file_flags(const void *buf, unsigned long len, enum object_type type, struct object_id *oid, struct object_id *comapt_oid_in, unsigned flags); diff --git a/read-cache.c b/read-cache.c index 2f9e21c897d..23028f43a11 100644 --- a/read-cache.c +++ b/read-cache.c @@ -706,11 +706,11 @@ int add_to_index(struct index_state *istate, const char *path, struct stat *st, int intent_only = flags & ADD_CACHE_INTENT; int add_option = (ADD_CACHE_OK_TO_ADD|ADD_CACHE_OK_TO_REPLACE| (intent_only ? ADD_CACHE_NEW_ONLY : 0)); - unsigned hash_flags = pretend ? 0 : HASH_WRITE_OBJECT; + unsigned hash_flags = pretend ? 0 : INDEX_WRITE_OBJECT; struct object_id oid; if (flags & ADD_CACHE_RENORMALIZE) - hash_flags |= HASH_RENORMALIZE; + hash_flags |= INDEX_RENORMALIZE; if (!S_ISREG(st_mode) && !S_ISLNK(st_mode) && !S_ISDIR(st_mode)) return error(_("%s: can only add regular files, symbolic links or git-directories"), path); From patchwork Tue Apr 8 10:24:14 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Patrick Steinhardt X-Patchwork-Id: 14042668 Received: from fhigh-a1-smtp.messagingengine.com (fhigh-a1-smtp.messagingengine.com [103.168.172.152]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 0EAA22690FE for ; Tue, 8 Apr 2025 10:24:19 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=103.168.172.152 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1744107862; cv=none; b=ZiwlVZbpxwgO3WoXNXL0eFMY5miq0wAxk8SLKmkbNr+Tx/Fej6DlwzMuXccs568c75F8Q1nw3ZIo5FkhwonVH09YxznjMHeKDVuFB2YbfI3pH1Q25VKtnTyuEZaAl1DXp9M00de5rJJx9Ja+daOABi44fmnEleL8v09bEui7VWc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1744107862; c=relaxed/simple; bh=6QuVBFSHCoLhz96SM9gpIWwkZ+1RMeQbCKqDeuyiOmU=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=Z57PwmYc2Yw+15VGkcoFUXPROeMzXH3MyIZf+rtiUxE3zh3v1qIxTwzeruOdaE8MBVxVIA5Cv386+8rHVgscZvCnWYHbAtKbRBr6VngrdgTFkad4V90ccGjYM4uEZrk/HJY72vRz5AASg4qSt29g3HhAwPiRTHVpzLI5kDPJQ6U= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=pks.im; spf=pass smtp.mailfrom=pks.im; dkim=pass (2048-bit key) header.d=pks.im header.i=@pks.im header.b=En4GnU9d; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b=au9Dx9lS; arc=none smtp.client-ip=103.168.172.152 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=pks.im Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=pks.im Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=pks.im header.i=@pks.im header.b="En4GnU9d"; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b="au9Dx9lS" Received: from phl-compute-01.internal (phl-compute-01.phl.internal [10.202.2.41]) by mailfhigh.phl.internal (Postfix) with ESMTP id E133711401E7 for ; Tue, 8 Apr 2025 06:24:18 -0400 (EDT) Received: from phl-mailfrontend-01 ([10.202.2.162]) by phl-compute-01.internal (MEProxy); Tue, 08 Apr 2025 06:24:18 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=pks.im; h=cc:cc :content-transfer-encoding:content-type:content-type:date:date :from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:subject:subject:to:to; s=fm2; t=1744107858; x=1744194258; bh=2bC4N96kesFadU9q5vk8Ry3GQ2L/6mo94EGYu9P2Y+E=; b= En4GnU9dLFmZmtSKrbXrmPTiHeOerCoByXiREff30w4TFSi5sAIZE11j4R+1W1fj yvD7uJJHeL5pZAYh/YmHpiik/z3gXUbDdw+ziEVLSkvGn3Fh6ntPhUDyXC7mqwkX BDs5LPLN/clwXaJYe/TAODdVeJ3ievrwHkieQNW3lvT2VAHnktz1ELYjZ33GnRnm 881z4uklFxjYh28YI7qI4tFJ6Dd/EPl/W/qEhALdRdCBB8BuJeuytEvbUD3VLrYD r+VRG62aCFE55PbqrOLUJ9j1+7b9D0YWOS4pl5khbB5IKISJq0rJG9QfaL4y+5YR aAro9xPBFO7NP+8eL6gl2Q== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-transfer-encoding :content-type:content-type:date:date:feedback-id:feedback-id :from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:subject:subject:to:to:x-me-proxy :x-me-sender:x-me-sender:x-sasl-enc; s=fm2; t=1744107858; x= 1744194258; bh=2bC4N96kesFadU9q5vk8Ry3GQ2L/6mo94EGYu9P2Y+E=; b=a u9Dx9lSY4uEmmQcb9/Oo1Wvu0RWwjcdAxmQKLYr1H/4hDYygHmbZ/Q0ksp7kJC8v MiwlL3KiDMqi3o6mdiCloqyr0TVP9fTO3w6FAB8T7HIJUIVmM3rHytMBnsUR1LgM Y2/migJQEkI40NXcb7sJ9PR+iW4Ay9URfFaw7TPV/TvMDBDlkmcueDXMEr1ekxIw gabgKwaQaquJSAkNsLuz2dHQcSgYj6v+3pd9Z2SHaUZTmvxtxedXZGiHrKWxWfeW s5ajZ0qXodFQEDw/QV3LxjnLDcsaM1dI8SZV9sQIYXgIfozaq3IBltUcUyVmhrrq X/jDify46JQJULjr/MDXA== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeefvddrtddtgddvtddvkeegucetufdoteggodetrf dotffvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdggtfgfnhhsuhgsshgtrhhisggv pdfurfetoffkrfgpnffqhgenuceurghilhhouhhtmecufedttdenucenucfjughrpefhff fugggtgffkfhgjvfevofesthejredtredtjeenucfhrhhomheprfgrthhrihgtkhcuufht vghinhhhrghrughtuceophhssehpkhhsrdhimheqnecuggftrfgrthhtvghrnhepffeuie dujedvkeehuedvkeefffeivdeuleetkeduheejteekgedvudfgtdfgieelnecuvehluhhs thgvrhfuihiivgepudenucfrrghrrghmpehmrghilhhfrhhomhepphhssehpkhhsrdhimh dpnhgspghrtghpthhtohepuddpmhhouggvpehsmhhtphhouhhtpdhrtghpthhtohepghhi thesvhhgvghrrdhkvghrnhgvlhdrohhrgh X-ME-Proxy: Feedback-ID: i197146af:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA for ; Tue, 8 Apr 2025 06:24:18 -0400 (EDT) Received: by vm-mail (OpenSMTPD) with ESMTPSA id 04f662f5 (TLSv1.3:TLS_AES_256_GCM_SHA384:256:NO) for ; Tue, 8 Apr 2025 10:24:17 +0000 (UTC) From: Patrick Steinhardt Date: Tue, 08 Apr 2025 12:24:14 +0200 Subject: [PATCH 6/9] object-file: split out functions relating to index subsystem Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20250408-pks-split-object-file-v1-6-f1fd50191143@pks.im> References: <20250408-pks-split-object-file-v1-0-f1fd50191143@pks.im> In-Reply-To: <20250408-pks-split-object-file-v1-0-f1fd50191143@pks.im> To: git@vger.kernel.org Cc: X-Mailer: b4 0.14.2 Split out functions relating to the index subsystem from "object-file.c" to help us separate concerns. Signed-off-by: Patrick Steinhardt --- builtin/difftool.c | 2 +- builtin/hash-object.c | 1 + builtin/replace.c | 1 + bulk-checkin.c | 1 + diff.c | 2 +- notes-merge.c | 1 + object-file.c | 225 -------------------------------------------------- object-file.h | 11 --- read-cache.c | 224 +++++++++++++++++++++++++++++++++++++++++++++++++ read-cache.h | 9 ++ 10 files changed, 239 insertions(+), 238 deletions(-) diff --git a/builtin/difftool.c b/builtin/difftool.c index 41cd00066cc..45d6ea8a801 100644 --- a/builtin/difftool.c +++ b/builtin/difftool.c @@ -22,7 +22,7 @@ #include "gettext.h" #include "hex.h" #include "parse-options.h" -#include "read-cache-ll.h" +#include "read-cache.h" #include "repository.h" #include "sparse-index.h" #include "strvec.h" diff --git a/builtin/hash-object.c b/builtin/hash-object.c index e7c0d6afdef..7aa889b1b4a 100644 --- a/builtin/hash-object.c +++ b/builtin/hash-object.c @@ -15,6 +15,7 @@ #include "blob.h" #include "quote.h" #include "parse-options.h" +#include "read-cache.h" #include "setup.h" #include "strbuf.h" #include "write-or-die.h" diff --git a/builtin/replace.c b/builtin/replace.c index 2b4fc9a68b3..ce2948a9b2a 100644 --- a/builtin/replace.c +++ b/builtin/replace.c @@ -20,6 +20,7 @@ #include "object-file.h" #include "object-name.h" #include "object-store-ll.h" +#include "read-cache.h" #include "replace-object.h" #include "tag.h" #include "wildmatch.h" diff --git a/bulk-checkin.c b/bulk-checkin.c index 309201a76a6..0b9bad92868 100644 --- a/bulk-checkin.c +++ b/bulk-checkin.c @@ -11,6 +11,7 @@ #include "gettext.h" #include "hex.h" #include "lockfile.h" +#include "read-cache.h" #include "repository.h" #include "csum-file.h" #include "pack.h" diff --git a/diff.c b/diff.c index 3bcf5028831..c361199ab1b 100644 --- a/diff.c +++ b/diff.c @@ -42,7 +42,7 @@ #include "dir.h" #include "object-file.h" #include "object-name.h" -#include "read-cache-ll.h" +#include "read-cache.h" #include "setup.h" #include "strmap.h" #include "ws.h" diff --git a/notes-merge.c b/notes-merge.c index d9f2914c819..13750490eff 100644 --- a/notes-merge.c +++ b/notes-merge.c @@ -10,6 +10,7 @@ #include "object-name.h" #include "object-store-ll.h" #include "path.h" +#include "read-cache.h" #include "repository.h" #include "diff.h" #include "diffcore.h" diff --git a/object-file.c b/object-file.c index dea5a70a331..235d29858ff 100644 --- a/object-file.c +++ b/object-file.c @@ -12,9 +12,7 @@ #include "git-compat-util.h" #include "bulk-checkin.h" -#include "convert.h" #include "environment.h" -#include "fsck.h" #include "gettext.h" #include "hex.h" #include "loose.h" @@ -25,22 +23,11 @@ #include "pack.h" #include "packfile.h" #include "path.h" -#include "setup.h" #include "streaming.h" /* The maximum size for an object header. */ #define MAX_HEADER_LEN 32 -static int get_conv_flags(unsigned flags) -{ - if (flags & INDEX_RENORMALIZE) - return CONV_EOL_RENORMALIZE; - else if (flags & INDEX_WRITE_OBJECT) - return global_conv_flags_eol | CONV_WRITE_OBJECT; - else - return 0; -} - static void fill_loose_path(struct strbuf *buf, const struct object_id *oid) { int i; @@ -1225,218 +1212,6 @@ int force_object_loose(const struct object_id *oid, time_t mtime) return ret; } -/* - * We can't use the normal fsck_error_function() for index_mem(), - * because we don't yet have a valid oid for it to report. Instead, - * report the minimal fsck error here, and rely on the caller to - * give more context. - */ -static int hash_format_check_report(struct fsck_options *opts UNUSED, - void *fsck_report UNUSED, - enum fsck_msg_type msg_type UNUSED, - enum fsck_msg_id msg_id UNUSED, - const char *message) -{ - error(_("object fails fsck: %s"), message); - return 1; -} - -static int index_mem(struct index_state *istate, - struct object_id *oid, - const void *buf, size_t size, - enum object_type type, - const char *path, unsigned flags) -{ - struct strbuf nbuf = STRBUF_INIT; - int ret = 0; - int write_object = flags & INDEX_WRITE_OBJECT; - - if (!type) - type = OBJ_BLOB; - - /* - * Convert blobs to git internal format - */ - if ((type == OBJ_BLOB) && path) { - if (convert_to_git(istate, path, buf, size, &nbuf, - get_conv_flags(flags))) { - buf = nbuf.buf; - size = nbuf.len; - } - } - if (flags & INDEX_FORMAT_CHECK) { - struct fsck_options opts = FSCK_OPTIONS_DEFAULT; - - opts.strict = 1; - opts.error_func = hash_format_check_report; - if (fsck_buffer(null_oid(the_hash_algo), type, buf, size, &opts)) - die(_("refusing to create malformed object")); - fsck_finish(&opts); - } - - if (write_object) - ret = write_object_file(buf, size, type, oid); - else - hash_object_file(the_hash_algo, buf, size, type, oid); - - strbuf_release(&nbuf); - return ret; -} - -static int index_stream_convert_blob(struct index_state *istate, - struct object_id *oid, - int fd, - const char *path, - unsigned flags) -{ - int ret = 0; - const int write_object = flags & INDEX_WRITE_OBJECT; - struct strbuf sbuf = STRBUF_INIT; - - assert(path); - assert(would_convert_to_git_filter_fd(istate, path)); - - convert_to_git_filter_fd(istate, path, fd, &sbuf, - get_conv_flags(flags)); - - if (write_object) - ret = write_object_file(sbuf.buf, sbuf.len, OBJ_BLOB, - oid); - else - hash_object_file(the_hash_algo, sbuf.buf, sbuf.len, OBJ_BLOB, - oid); - strbuf_release(&sbuf); - return ret; -} - -static int index_pipe(struct index_state *istate, struct object_id *oid, - int fd, enum object_type type, - const char *path, unsigned flags) -{ - struct strbuf sbuf = STRBUF_INIT; - int ret; - - if (strbuf_read(&sbuf, fd, 4096) >= 0) - ret = index_mem(istate, oid, sbuf.buf, sbuf.len, type, path, flags); - else - ret = -1; - strbuf_release(&sbuf); - return ret; -} - -#define SMALL_FILE_SIZE (32*1024) - -static int index_core(struct index_state *istate, - struct object_id *oid, int fd, size_t size, - enum object_type type, const char *path, - unsigned flags) -{ - int ret; - - if (!size) { - ret = index_mem(istate, oid, "", size, type, path, flags); - } else if (size <= SMALL_FILE_SIZE) { - char *buf = xmalloc(size); - ssize_t read_result = read_in_full(fd, buf, size); - if (read_result < 0) - ret = error_errno(_("read error while indexing %s"), - path ? path : ""); - else if (read_result != size) - ret = error(_("short read while indexing %s"), - path ? path : ""); - else - ret = index_mem(istate, oid, buf, size, type, path, flags); - free(buf); - } else { - void *buf = xmmap(NULL, size, PROT_READ, MAP_PRIVATE, fd, 0); - ret = index_mem(istate, oid, buf, size, type, path, flags); - munmap(buf, size); - } - return ret; -} - -/* - * This creates one packfile per large blob unless bulk-checkin - * machinery is "plugged". - * - * This also bypasses the usual "convert-to-git" dance, and that is on - * purpose. We could write a streaming version of the converting - * functions and insert that before feeding the data to fast-import - * (or equivalent in-core API described above). However, that is - * somewhat complicated, as we do not know the size of the filter - * result, which we need to know beforehand when writing a git object. - * Since the primary motivation for trying to stream from the working - * tree file and to avoid mmaping it in core is to deal with large - * binary blobs, they generally do not want to get any conversion, and - * callers should avoid this code path when filters are requested. - */ -static int index_blob_stream(struct object_id *oid, int fd, size_t size, - const char *path, - unsigned flags) -{ - return index_blob_bulk_checkin(oid, fd, size, path, flags); -} - -int index_fd(struct index_state *istate, struct object_id *oid, - int fd, struct stat *st, - enum object_type type, const char *path, unsigned flags) -{ - int ret; - - /* - * Call xsize_t() only when needed to avoid potentially unnecessary - * die() for large files. - */ - if (type == OBJ_BLOB && path && would_convert_to_git_filter_fd(istate, path)) - ret = index_stream_convert_blob(istate, oid, fd, path, flags); - else if (!S_ISREG(st->st_mode)) - ret = index_pipe(istate, oid, fd, type, path, flags); - else if (st->st_size <= repo_settings_get_big_file_threshold(the_repository) || - type != OBJ_BLOB || - (path && would_convert_to_git(istate, path))) - ret = index_core(istate, oid, fd, xsize_t(st->st_size), - type, path, flags); - else - ret = index_blob_stream(oid, fd, xsize_t(st->st_size), path, - flags); - close(fd); - return ret; -} - -int index_path(struct index_state *istate, struct object_id *oid, - const char *path, struct stat *st, unsigned flags) -{ - int fd; - struct strbuf sb = STRBUF_INIT; - int rc = 0; - - switch (st->st_mode & S_IFMT) { - case S_IFREG: - fd = open(path, O_RDONLY); - if (fd < 0) - return error_errno("open(\"%s\")", path); - if (index_fd(istate, oid, fd, st, OBJ_BLOB, path, flags) < 0) - return error(_("%s: failed to insert into database"), - path); - break; - case S_IFLNK: - if (strbuf_readlink(&sb, path, st->st_size)) - return error_errno("readlink(\"%s\")", path); - if (!(flags & INDEX_WRITE_OBJECT)) - hash_object_file(the_hash_algo, sb.buf, sb.len, - OBJ_BLOB, oid); - else if (write_object_file(sb.buf, sb.len, OBJ_BLOB, oid)) - rc = error(_("%s: failed to insert into database"), path); - strbuf_release(&sb); - break; - case S_IFDIR: - return repo_resolve_gitlink_ref(the_repository, path, "HEAD", oid); - default: - return error(_("%s: unsupported file type"), path); - } - return rc; -} - int read_pack_header(int fd, struct pack_header *header) { if (read_in_full(fd, header, sizeof(*header)) != sizeof(*header)) diff --git a/object-file.h b/object-file.h index c002fbe2345..7d48994f1b6 100644 --- a/object-file.h +++ b/object-file.h @@ -4,8 +4,6 @@ #include "git-zlib.h" #include "object.h" -struct index_state; - /* * Set this to 0 to prevent oid_object_info_extended() from fetching missing * blobs. This has a difference only if extensions.partialClone is set. @@ -14,15 +12,6 @@ struct index_state; */ extern int fetch_if_missing; -enum { - INDEX_WRITE_OBJECT = (1 << 0), - INDEX_FORMAT_CHECK = (1 << 1), - INDEX_RENORMALIZE = (1 << 2), -}; - -int index_fd(struct index_state *istate, struct object_id *oid, int fd, struct stat *st, enum object_type type, const char *path, unsigned flags); -int index_path(struct index_state *istate, struct object_id *oid, const char *path, struct stat *st, unsigned flags); - struct object_directory; const char *odb_loose_path(struct object_directory *odb, diff --git a/read-cache.c b/read-cache.c index 23028f43a11..9909b56902f 100644 --- a/read-cache.c +++ b/read-cache.c @@ -10,9 +10,11 @@ #include "git-compat-util.h" #include "bulk-checkin.h" #include "config.h" +#include "convert.h" #include "date.h" #include "diff.h" #include "diffcore.h" +#include "fsck.h" #include "hex.h" #include "tempfile.h" #include "lockfile.h" @@ -4007,3 +4009,225 @@ int add_files_to_cache(struct repository *repo, const char *prefix, release_revisions(&rev); return !!data.add_errors; } + +static int get_conv_flags(unsigned flags) +{ + if (flags & INDEX_RENORMALIZE) + return CONV_EOL_RENORMALIZE; + else if (flags & INDEX_WRITE_OBJECT) + return global_conv_flags_eol | CONV_WRITE_OBJECT; + else + return 0; +} + +/* + * We can't use the normal fsck_error_function() for index_mem(), + * because we don't yet have a valid oid for it to report. Instead, + * report the minimal fsck error here, and rely on the caller to + * give more context. + */ +static int hash_format_check_report(struct fsck_options *opts UNUSED, + void *fsck_report UNUSED, + enum fsck_msg_type msg_type UNUSED, + enum fsck_msg_id msg_id UNUSED, + const char *message) +{ + error(_("object fails fsck: %s"), message); + return 1; +} + +static int index_mem(struct index_state *istate, + struct object_id *oid, + const void *buf, size_t size, + enum object_type type, + const char *path, unsigned flags) +{ + struct strbuf nbuf = STRBUF_INIT; + int ret = 0; + int write_object = flags & INDEX_WRITE_OBJECT; + + if (!type) + type = OBJ_BLOB; + + /* + * Convert blobs to git internal format + */ + if ((type == OBJ_BLOB) && path) { + if (convert_to_git(istate, path, buf, size, &nbuf, + get_conv_flags(flags))) { + buf = nbuf.buf; + size = nbuf.len; + } + } + if (flags & INDEX_FORMAT_CHECK) { + struct fsck_options opts = FSCK_OPTIONS_DEFAULT; + + opts.strict = 1; + opts.error_func = hash_format_check_report; + if (fsck_buffer(null_oid(the_hash_algo), type, buf, size, &opts)) + die(_("refusing to create malformed object")); + fsck_finish(&opts); + } + + if (write_object) + ret = write_object_file(buf, size, type, oid); + else + hash_object_file(the_hash_algo, buf, size, type, oid); + + strbuf_release(&nbuf); + return ret; +} + +static int index_stream_convert_blob(struct index_state *istate, + struct object_id *oid, + int fd, + const char *path, + unsigned flags) +{ + int ret = 0; + const int write_object = flags & INDEX_WRITE_OBJECT; + struct strbuf sbuf = STRBUF_INIT; + + assert(path); + assert(would_convert_to_git_filter_fd(istate, path)); + + convert_to_git_filter_fd(istate, path, fd, &sbuf, + get_conv_flags(flags)); + + if (write_object) + ret = write_object_file(sbuf.buf, sbuf.len, OBJ_BLOB, + oid); + else + hash_object_file(the_hash_algo, sbuf.buf, sbuf.len, OBJ_BLOB, + oid); + strbuf_release(&sbuf); + return ret; +} + +static int index_pipe(struct index_state *istate, struct object_id *oid, + int fd, enum object_type type, + const char *path, unsigned flags) +{ + struct strbuf sbuf = STRBUF_INIT; + int ret; + + if (strbuf_read(&sbuf, fd, 4096) >= 0) + ret = index_mem(istate, oid, sbuf.buf, sbuf.len, type, path, flags); + else + ret = -1; + strbuf_release(&sbuf); + return ret; +} + +#define SMALL_FILE_SIZE (32*1024) + +static int index_core(struct index_state *istate, + struct object_id *oid, int fd, size_t size, + enum object_type type, const char *path, + unsigned flags) +{ + int ret; + + if (!size) { + ret = index_mem(istate, oid, "", size, type, path, flags); + } else if (size <= SMALL_FILE_SIZE) { + char *buf = xmalloc(size); + ssize_t read_result = read_in_full(fd, buf, size); + if (read_result < 0) + ret = error_errno(_("read error while indexing %s"), + path ? path : ""); + else if (read_result != size) + ret = error(_("short read while indexing %s"), + path ? path : ""); + else + ret = index_mem(istate, oid, buf, size, type, path, flags); + free(buf); + } else { + void *buf = xmmap(NULL, size, PROT_READ, MAP_PRIVATE, fd, 0); + ret = index_mem(istate, oid, buf, size, type, path, flags); + munmap(buf, size); + } + return ret; +} + +/* + * This creates one packfile per large blob unless bulk-checkin + * machinery is "plugged". + * + * This also bypasses the usual "convert-to-git" dance, and that is on + * purpose. We could write a streaming version of the converting + * functions and insert that before feeding the data to fast-import + * (or equivalent in-core API described above). However, that is + * somewhat complicated, as we do not know the size of the filter + * result, which we need to know beforehand when writing a git object. + * Since the primary motivation for trying to stream from the working + * tree file and to avoid mmaping it in core is to deal with large + * binary blobs, they generally do not want to get any conversion, and + * callers should avoid this code path when filters are requested. + */ +static int index_blob_stream(struct object_id *oid, int fd, size_t size, + const char *path, + unsigned flags) +{ + return index_blob_bulk_checkin(oid, fd, size, path, flags); +} + +int index_fd(struct index_state *istate, struct object_id *oid, + int fd, struct stat *st, + enum object_type type, const char *path, unsigned flags) +{ + int ret; + + /* + * Call xsize_t() only when needed to avoid potentially unnecessary + * die() for large files. + */ + if (type == OBJ_BLOB && path && would_convert_to_git_filter_fd(istate, path)) + ret = index_stream_convert_blob(istate, oid, fd, path, flags); + else if (!S_ISREG(st->st_mode)) + ret = index_pipe(istate, oid, fd, type, path, flags); + else if (st->st_size <= repo_settings_get_big_file_threshold(the_repository) || + type != OBJ_BLOB || + (path && would_convert_to_git(istate, path))) + ret = index_core(istate, oid, fd, xsize_t(st->st_size), + type, path, flags); + else + ret = index_blob_stream(oid, fd, xsize_t(st->st_size), path, + flags); + close(fd); + return ret; +} + +int index_path(struct index_state *istate, struct object_id *oid, + const char *path, struct stat *st, unsigned flags) +{ + int fd; + struct strbuf sb = STRBUF_INIT; + int rc = 0; + + switch (st->st_mode & S_IFMT) { + case S_IFREG: + fd = open(path, O_RDONLY); + if (fd < 0) + return error_errno("open(\"%s\")", path); + if (index_fd(istate, oid, fd, st, OBJ_BLOB, path, flags) < 0) + return error(_("%s: failed to insert into database"), + path); + break; + case S_IFLNK: + if (strbuf_readlink(&sb, path, st->st_size)) + return error_errno("readlink(\"%s\")", path); + if (!(flags & INDEX_WRITE_OBJECT)) + hash_object_file(the_hash_algo, sb.buf, sb.len, + OBJ_BLOB, oid); + else if (write_object_file(sb.buf, sb.len, OBJ_BLOB, oid)) + rc = error(_("%s: failed to insert into database"), path); + strbuf_release(&sb); + break; + case S_IFDIR: + return repo_resolve_gitlink_ref(the_repository, path, "HEAD", oid); + default: + return error(_("%s: unsupported file type"), path); + } + return rc; +} diff --git a/read-cache.h b/read-cache.h index 043da1f1aae..d655eb64bb7 100644 --- a/read-cache.h +++ b/read-cache.h @@ -42,4 +42,13 @@ static inline int ce_path_match(struct index_state *istate, S_ISDIR(ce->ce_mode) || S_ISGITLINK(ce->ce_mode)); } +enum { + INDEX_WRITE_OBJECT = (1 << 0), + INDEX_FORMAT_CHECK = (1 << 1), + INDEX_RENORMALIZE = (1 << 2), +}; + +int index_fd(struct index_state *istate, struct object_id *oid, int fd, struct stat *st, enum object_type type, const char *path, unsigned flags); +int index_path(struct index_state *istate, struct object_id *oid, const char *path, struct stat *st, unsigned flags); + #endif /* READ_CACHE_H */ From patchwork Tue Apr 8 10:24:15 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Patrick Steinhardt X-Patchwork-Id: 14042667 Received: from fout-a6-smtp.messagingengine.com (fout-a6-smtp.messagingengine.com [103.168.172.149]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A6F1C269816 for ; Tue, 8 Apr 2025 10:24:20 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=103.168.172.149 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1744107862; cv=none; b=On5jKS3qKJIxXEaszm+euMjTPkqt6MdO+SVrHVJ+G0jMy4Op03cyCqpqjGk+ApRArf0qZNzujgXKGAezi/eBERJD2vLPgHfCbf5s5osH+2xIy7OKqIsIzrBCeKPxcfpCzcKMuFiBGtPellTjxjXIC5O1UT6BX+G3+3y9tSBGk6Y= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1744107862; c=relaxed/simple; bh=tdo6N7gYBZ7OqLLCVpQo/YPaNPAJkIOxbem5VdrHlP4=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=sIN9cGk8UJuQK6xDihYqxsD7vlC68KaUVAS6Dp+xjFLV52IrT+DMsJr51vgQIfJoL1V01IsUrrrmuvOCKhdsTej4DaHw4NVNoK5ViFOyyrm24gNSf4kso+B+VwRW09LV2ujSxE2zksICPpZgvgkDS3ST7CrcF2kk/WnIIoJvmQs= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=pks.im; spf=pass smtp.mailfrom=pks.im; dkim=pass (2048-bit key) header.d=pks.im header.i=@pks.im header.b=atokkVt4; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b=ua/V9rcD; arc=none smtp.client-ip=103.168.172.149 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=pks.im Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=pks.im Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=pks.im header.i=@pks.im header.b="atokkVt4"; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b="ua/V9rcD" Received: from phl-compute-01.internal (phl-compute-01.phl.internal [10.202.2.41]) by mailfout.phl.internal (Postfix) with ESMTP id 9CCD1138014C for ; Tue, 8 Apr 2025 06:24:19 -0400 (EDT) Received: from phl-mailfrontend-01 ([10.202.2.162]) by phl-compute-01.internal (MEProxy); Tue, 08 Apr 2025 06:24:19 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=pks.im; h=cc:cc :content-transfer-encoding:content-type:content-type:date:date :from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:subject:subject:to:to; s=fm2; t=1744107859; x=1744194259; bh=uz0gpbfH8pa8RfQWcSxNagIcdSRtD6lo8SKsXhp8C4g=; b= atokkVt4x2kHPrBjV4ns5ipLVKcctl2YWimFY4TcofpZo3uAg/lYLf4FkekeYkuS j48c3pRpg0qb+8Z4V+WZH6NlgL6Bvmk593TQ7KDvZBPawDdvMnvR7SK7sq/zYJHj 9eQMMUrjKYa+XFxuvYo12JLBbtNVk3buVZDz3DYBxGcosOc+OU7tFgyLJpcJgZDO VVNDPNzVZbYQ0kKgs/yF9GxMbWGmnfpkFMwBCi092uchZV5oI2SRCkQfRWkP7EBI 7s8rQbWjYQ7rCKZUwic8tcX92RDKODJlsbCqMon3mtyO3d2Zalu8hQE1hvOOD3CU VU9Y1LQLOCBJEfce5bEcPw== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-transfer-encoding :content-type:content-type:date:date:feedback-id:feedback-id :from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:subject:subject:to:to:x-me-proxy :x-me-sender:x-me-sender:x-sasl-enc; s=fm2; t=1744107859; x= 1744194259; bh=uz0gpbfH8pa8RfQWcSxNagIcdSRtD6lo8SKsXhp8C4g=; b=u a/V9rcDtkS4XSFZxxoUYfwLQt/HKybRh+n6QBJo0oxFN+WM0FWz/z/ry+giA75rt YoEtNin8g4+TP7fOUI6NSNP3CuERDx033d8WOyaz5WtDk9k2XuwFq7Xqg3CDY4JW wKrBvvsqjNQE4tB1Jm3CHuudzaNs4blkWE3xJ5ncECuY+5fwGennd8UMbzGiSZnm jCXJxwHz98ZLGMjPlXDoEKuG5Pan9nN3QFYIefw0DGtRfer2/iJoJg2etbe2Xsaf BLNvHmVRTtPvpwGOO3SGhgl/uYGrNiWzSCMdzBR8Ow6LKYtva/idKIMZ6TyZZQyx 929e2jZCse8r0Fstt1zjw== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeefvddrtddtgddvtddvkeegucetufdoteggodetrf dotffvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdggtfgfnhhsuhgsshgtrhhisggv pdfurfetoffkrfgpnffqhgenuceurghilhhouhhtmecufedttdenucenucfjughrpefhff fugggtgffkfhgjvfevofesthejredtredtjeenucfhrhhomheprfgrthhrihgtkhcuufht vghinhhhrghrughtuceophhssehpkhhsrdhimheqnecuggftrfgrthhtvghrnhepffeuie dujedvkeehuedvkeefffeivdeuleetkeduheejteekgedvudfgtdfgieelnecuvehluhhs thgvrhfuihiivgepudenucfrrghrrghmpehmrghilhhfrhhomhepphhssehpkhhsrdhimh dpnhgspghrtghpthhtohepuddpmhhouggvpehsmhhtphhouhhtpdhrtghpthhtohepghhi thesvhhgvghrrdhkvghrnhgvlhdrohhrgh X-ME-Proxy: Feedback-ID: i197146af:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA for ; Tue, 8 Apr 2025 06:24:19 -0400 (EDT) Received: by vm-mail (OpenSMTPD) with ESMTPSA id fcb0d047 (TLSv1.3:TLS_AES_256_GCM_SHA384:256:NO) for ; Tue, 8 Apr 2025 10:24:18 +0000 (UTC) From: Patrick Steinhardt Date: Tue, 08 Apr 2025 12:24:15 +0200 Subject: [PATCH 7/9] object: split out functions relating to object store subsystem Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20250408-pks-split-object-file-v1-7-f1fd50191143@pks.im> References: <20250408-pks-split-object-file-v1-0-f1fd50191143@pks.im> In-Reply-To: <20250408-pks-split-object-file-v1-0-f1fd50191143@pks.im> To: git@vger.kernel.org Cc: X-Mailer: b4 0.14.2 Split out functions relating to the object store subsystem from "object.c". This helps us to separate concerns. Signed-off-by: Patrick Steinhardt --- object-store-ll.h | 3 --- object-store.c | 66 ++++++++++++++++++++++++++++++++++++++++++++++++++++++ object.c | 67 ------------------------------------------------------- 3 files changed, 66 insertions(+), 70 deletions(-) diff --git a/object-store-ll.h b/object-store-ll.h index 8ae80b8a5fa..8bb0f33f9a8 100644 --- a/object-store-ll.h +++ b/object-store-ll.h @@ -92,9 +92,6 @@ struct oidtree *odb_loose_cache(struct object_directory *odb, /* Empty the loose object cache for the specified object directory. */ void odb_clear_loose_cache(struct object_directory *odb); -/* Clear and free the specified object directory */ -void free_object_directory(struct object_directory *odb); - struct packed_git { struct hashmap_entry packmap_ent; struct packed_git *next; diff --git a/object-store.c b/object-store.c index e5f1f00cdde..ea2d86c429b 100644 --- a/object-store.c +++ b/object-store.c @@ -2,11 +2,13 @@ #include "git-compat-util.h" #include "abspath.h" +#include "commit-graph.h" #include "config.h" #include "environment.h" #include "gettext.h" #include "hex.h" #include "lockfile.h" +#include "loose.h" #include "object-file-convert.h" #include "object-file.h" #include "object-store.h" @@ -361,6 +363,14 @@ struct object_directory *set_temporary_primary_odb(const char *dir, int will_des return new_odb->next; } +static void free_object_directory(struct object_directory *odb) +{ + free(odb->path); + odb_clear_loose_cache(odb); + loose_object_map_clear(&odb->loose_map); + free(odb); +} + void restore_primary_odb(struct object_directory *restore_odb, const char *old_path) { struct object_directory *cur_odb = the_repository->objects->odb; @@ -970,3 +980,59 @@ void assert_oid_type(const struct object_id *oid, enum object_type expect) die(_("%s is not a valid '%s' object"), oid_to_hex(oid), type_name(expect)); } + +struct raw_object_store *raw_object_store_new(void) +{ + struct raw_object_store *o = xmalloc(sizeof(*o)); + + memset(o, 0, sizeof(*o)); + INIT_LIST_HEAD(&o->packed_git_mru); + hashmap_init(&o->pack_map, pack_map_entry_cmp, NULL, 0); + pthread_mutex_init(&o->replace_mutex, NULL); + return o; +} + +static void free_object_directories(struct raw_object_store *o) +{ + while (o->odb) { + struct object_directory *next; + + next = o->odb->next; + free_object_directory(o->odb); + o->odb = next; + } + kh_destroy_odb_path_map(o->odb_by_path); + o->odb_by_path = NULL; +} + +void raw_object_store_clear(struct raw_object_store *o) +{ + FREE_AND_NULL(o->alternate_db); + + oidmap_free(o->replace_map, 1); + FREE_AND_NULL(o->replace_map); + pthread_mutex_destroy(&o->replace_mutex); + + free_commit_graph(o->commit_graph); + o->commit_graph = NULL; + o->commit_graph_attempted = 0; + + free_object_directories(o); + o->odb_tail = NULL; + o->loaded_alternates = 0; + + INIT_LIST_HEAD(&o->packed_git_mru); + close_object_store(o); + + /* + * `close_object_store()` only closes the packfiles, but doesn't free + * them. We thus have to do this manually. + */ + for (struct packed_git *p = o->packed_git, *next; p; p = next) { + next = p->next; + free(p); + } + o->packed_git = NULL; + + hashmap_clear(&o->pack_map); +} diff --git a/object.c b/object.c index 154525a4972..ccda798b75f 100644 --- a/object.c +++ b/object.c @@ -6,16 +6,13 @@ #include "object.h" #include "replace-object.h" #include "object-file.h" -#include "object-store.h" #include "blob.h" #include "statinfo.h" #include "tree.h" #include "commit.h" #include "tag.h" #include "alloc.h" -#include "packfile.h" #include "commit-graph.h" -#include "loose.h" unsigned int get_max_object_index(const struct repository *repo) { @@ -567,70 +564,6 @@ struct parsed_object_pool *parsed_object_pool_new(struct repository *repo) return o; } -struct raw_object_store *raw_object_store_new(void) -{ - struct raw_object_store *o = xmalloc(sizeof(*o)); - - memset(o, 0, sizeof(*o)); - INIT_LIST_HEAD(&o->packed_git_mru); - hashmap_init(&o->pack_map, pack_map_entry_cmp, NULL, 0); - pthread_mutex_init(&o->replace_mutex, NULL); - return o; -} - -void free_object_directory(struct object_directory *odb) -{ - free(odb->path); - odb_clear_loose_cache(odb); - loose_object_map_clear(&odb->loose_map); - free(odb); -} - -static void free_object_directories(struct raw_object_store *o) -{ - while (o->odb) { - struct object_directory *next; - - next = o->odb->next; - free_object_directory(o->odb); - o->odb = next; - } - kh_destroy_odb_path_map(o->odb_by_path); - o->odb_by_path = NULL; -} - -void raw_object_store_clear(struct raw_object_store *o) -{ - FREE_AND_NULL(o->alternate_db); - - oidmap_free(o->replace_map, 1); - FREE_AND_NULL(o->replace_map); - pthread_mutex_destroy(&o->replace_mutex); - - free_commit_graph(o->commit_graph); - o->commit_graph = NULL; - o->commit_graph_attempted = 0; - - free_object_directories(o); - o->odb_tail = NULL; - o->loaded_alternates = 0; - - INIT_LIST_HEAD(&o->packed_git_mru); - close_object_store(o); - - /* - * `close_object_store()` only closes the packfiles, but doesn't free - * them. We thus have to do this manually. - */ - for (struct packed_git *p = o->packed_git, *next; p; p = next) { - next = p->next; - free(p); - } - o->packed_git = NULL; - - hashmap_clear(&o->pack_map); -} - void parsed_object_pool_reset_commit_grafts(struct parsed_object_pool *o) { for (int i = 0; i < o->grafts_nr; i++) { From patchwork Tue Apr 8 10:24:16 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Patrick Steinhardt X-Patchwork-Id: 14042670 Received: from fhigh-a1-smtp.messagingengine.com (fhigh-a1-smtp.messagingengine.com [103.168.172.152]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 6825E269836 for ; Tue, 8 Apr 2025 10:24:21 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=103.168.172.152 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1744107864; cv=none; b=CX1vCRzytVILrJvtB8UuWaCnbmun8Ivf/ZhxoAHc8JIyLmh690OiuwNvcM/M1JtmJ58I//KLr+miOUMo4Spt40Q3uSZneWsVcJPJmNmYJLcJBZAGPbezks4vJf7/cZWziDlKbkpCjVSbNiGwmbfhpuP8+eAEZWx/QiQOGc2tA9U= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1744107864; c=relaxed/simple; bh=sfeHwrLRUymAkKZ71+ezIKMsThUL86dGgv5jRpRdTpQ=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=umlqdk0swzLAGeDOcK0URqH/WmrKTKSz0YRIH0RCRYdYGYE2AKK0la7F2t5h04QASCfRFILH0qZcVQQPLfuGPKchNBq8Iocg5JHrMZIoBYp9ze9HmZUpKFtZyN/cfCiDwiC4Q4hfAoI62ZpzLQPb5ug4p/Y716BZBU5JFpXrYC4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=pks.im; spf=pass smtp.mailfrom=pks.im; dkim=pass (2048-bit key) header.d=pks.im header.i=@pks.im header.b=NTMdprsX; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b=G5xDLfLT; arc=none smtp.client-ip=103.168.172.152 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=pks.im Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=pks.im Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=pks.im header.i=@pks.im header.b="NTMdprsX"; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b="G5xDLfLT" Received: from phl-compute-07.internal (phl-compute-07.phl.internal [10.202.2.47]) by mailfhigh.phl.internal (Postfix) with ESMTP id 6E0791140209 for ; Tue, 8 Apr 2025 06:24:20 -0400 (EDT) Received: from phl-mailfrontend-01 ([10.202.2.162]) by phl-compute-07.internal (MEProxy); Tue, 08 Apr 2025 06:24:20 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=pks.im; h=cc:cc :content-transfer-encoding:content-type:content-type:date:date :from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:subject:subject:to:to; s=fm2; t=1744107860; x=1744194260; bh=RLJs02eWR/X+Is+t4UZb6+R5uqM43mbhImirMOh8Xc4=; b= NTMdprsXIe5ODOlnNcMMdM8RCD5LAtGTxH06UbFFI6NrZAuwNTYUMwfEmJonMQTx TLJkU7Z2FvgkmYtVFJznFAxaAqJoExqevnEjWcBSpMEGd7eZA2VGL3TrejVJja1N p+JIVa2it0x0YtWbGtd8r3ZA8bAXqnJwBfSL28CNXc5wzukoU/tMZZtpTKeDnldr SPHKmvQ/G2aierAyOxdebX5zGepF4gpalBiGjVXg39/LibZ9/TfQ68IkbJ7vPZHh pqhoQG/t5UWrGNRX6qASVvsfFV7JTH7Lce9Xlr/xfeCyuSoN+9EaXVPf8hf9vlde +QyyIk+03EZDlWltabUALA== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-transfer-encoding :content-type:content-type:date:date:feedback-id:feedback-id :from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:subject:subject:to:to:x-me-proxy :x-me-sender:x-me-sender:x-sasl-enc; s=fm2; t=1744107860; x= 1744194260; bh=RLJs02eWR/X+Is+t4UZb6+R5uqM43mbhImirMOh8Xc4=; b=G 5xDLfLTv4tnfxcc2gHpAbL+rZ3eNp/OYgZOcVMn4BaHymyKi+bZpO1F2hB8X2Qz8 Z1x/I+KmlADqiXSzQig8igyu1aUULTDY5RwmmgYZIAmyjDQlMQ+67WrdwDoq/o/s Uy4Xa3mdEphfXNe/16A2TpNPQSnzZ5Qtm1Ob8quGeDYLjpMKEmSLSmwL0QQ2sxGd jr2RnCsjgW6fowydynhBLawHT7/kKlRAGl282YGft3I7Sc2cGECS/USaLqKyLpxV 3ey5i7hj2nsJ9ZhkIxQ9gYQCB5PCAlxA+x7ExkjYk14/55m/4LJIdChoeHdkgKIY /P86kze+/nlq+jOZEeoAQ== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeefvddrtddtgddvtddvkeegucetufdoteggodetrf dotffvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdggtfgfnhhsuhgsshgtrhhisggv pdfurfetoffkrfgpnffqhgenuceurghilhhouhhtmecufedttdenucenucfjughrpefhff fugggtgffkfhgjvfevofesthejredtredtjeenucfhrhhomheprfgrthhrihgtkhcuufht vghinhhhrghrughtuceophhssehpkhhsrdhimheqnecuggftrfgrthhtvghrnhepffeuie dujedvkeehuedvkeefffeivdeuleetkeduheejteekgedvudfgtdfgieelnecuvehluhhs thgvrhfuihiivgeptdenucfrrghrrghmpehmrghilhhfrhhomhepphhssehpkhhsrdhimh dpnhgspghrtghpthhtohepuddpmhhouggvpehsmhhtphhouhhtpdhrtghpthhtohepghhi thesvhhgvghrrdhkvghrnhgvlhdrohhrgh X-ME-Proxy: Feedback-ID: i197146af:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA for ; Tue, 8 Apr 2025 06:24:19 -0400 (EDT) Received: by vm-mail (OpenSMTPD) with ESMTPSA id 0e056cb3 (TLSv1.3:TLS_AES_256_GCM_SHA384:256:NO) for ; Tue, 8 Apr 2025 10:24:19 +0000 (UTC) From: Patrick Steinhardt Date: Tue, 08 Apr 2025 12:24:16 +0200 Subject: [PATCH 8/9] object-store: remove global array of cached objects Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20250408-pks-split-object-file-v1-8-f1fd50191143@pks.im> References: <20250408-pks-split-object-file-v1-0-f1fd50191143@pks.im> In-Reply-To: <20250408-pks-split-object-file-v1-0-f1fd50191143@pks.im> To: git@vger.kernel.org Cc: X-Mailer: b4 0.14.2 Cached objects are virtual objects that can be set up without writing anything into the object store directly. This mechanism for example allows us to create fake commits in git-blame(1). The cached objects are stored in a global variable. Refactor the code so that we instead store the array as part of the raw object store. This is another step into the direction of libifying our object database. Signed-off-by: Patrick Steinhardt --- blame.c | 2 +- object-store-ll.h | 14 +++++++++++++- object-store.c | 39 +++++++++++++++++++++++---------------- 3 files changed, 37 insertions(+), 18 deletions(-) diff --git a/blame.c b/blame.c index 703dab43e78..b7c5bd692e6 100644 --- a/blame.c +++ b/blame.c @@ -277,7 +277,7 @@ static struct commit *fake_working_tree_commit(struct repository *r, convert_to_git(r->index, path, buf.buf, buf.len, &buf, 0); origin->file.ptr = buf.buf; origin->file.size = buf.len; - pretend_object_file(buf.buf, buf.len, OBJ_BLOB, &origin->blob_oid); + pretend_object_file(the_repository, buf.buf, buf.len, OBJ_BLOB, &origin->blob_oid); /* * Read the current index, replace the path entry with diff --git a/object-store-ll.h b/object-store-ll.h index 8bb0f33f9a8..bb5e8798a1b 100644 --- a/object-store-ll.h +++ b/object-store-ll.h @@ -151,6 +151,8 @@ static inline int pack_map_entry_cmp(const void *cmp_data UNUSED, return strcmp(pg1->pack_name, key ? key : pg2->pack_name); } +struct cached_object_entry; + struct raw_object_store { /* * Set of all object directories; the main directory is first (and @@ -203,6 +205,15 @@ struct raw_object_store { unsigned flags; } kept_pack_cache; + /* + * This is meant to hold a *small* number of objects that you would + * want repo_read_object_file() to be able to return, but yet you do not want + * to write them into the object store (e.g. a browse-only + * application). + */ + struct cached_object_entry *cached_objects; + size_t cached_object_nr, cached_object_alloc; + /* * A map of packfiles to packed_git structs for tracking which * packs have been loaded already. @@ -272,7 +283,8 @@ void hash_object_file(const struct git_hash_algo *algo, const void *buf, * object in persistent storage before writing any other new objects * that reference it. */ -int pretend_object_file(void *, unsigned long, enum object_type, +int pretend_object_file(struct repository *repo, + void *buf, unsigned long len, enum object_type type, struct object_id *oid); struct object_info { diff --git a/object-store.c b/object-store.c index ea2d86c429b..17fa06a86fa 100644 --- a/object-store.c +++ b/object-store.c @@ -30,31 +30,31 @@ * to write them into the object store (e.g. a browse-only * application). */ -static struct cached_object_entry { +struct cached_object_entry { struct object_id oid; struct cached_object { enum object_type type; const void *buf; unsigned long size; } value; -} *cached_objects; -static int cached_object_nr, cached_object_alloc; +}; -static const struct cached_object *find_cached_object(const struct object_id *oid) +static const struct cached_object *find_cached_object(struct raw_object_store *object_store, + const struct object_id *oid) { static const struct cached_object empty_tree = { .type = OBJ_TREE, .buf = "", }; - int i; - const struct cached_object_entry *co = cached_objects; + const struct cached_object_entry *co = object_store->cached_objects; - for (i = 0; i < cached_object_nr; i++, co++) { + for (size_t i = 0; i < object_store->cached_object_nr; i++, co++) if (oideq(&co->oid, oid)) return &co->value; - } - if (oideq(oid, the_hash_algo->empty_tree)) + + if (oid->algo && oideq(oid, hash_algos[oid->algo].empty_tree)) return &empty_tree; + return NULL; } @@ -650,7 +650,7 @@ static int do_oid_object_info_extended(struct repository *r, if (!oi) oi = &blank_oi; - co = find_cached_object(real); + co = find_cached_object(r->objects, real); if (co) { if (oi->typep) *(oi->typep) = co->type; @@ -853,18 +853,21 @@ int oid_object_info(struct repository *r, return type; } -int pretend_object_file(void *buf, unsigned long len, enum object_type type, +int pretend_object_file(struct repository *repo, + void *buf, unsigned long len, enum object_type type, struct object_id *oid) { struct cached_object_entry *co; char *co_buf; - hash_object_file(the_hash_algo, buf, len, type, oid); - if (repo_has_object_file_with_flags(the_repository, oid, OBJECT_INFO_QUICK | OBJECT_INFO_SKIP_FETCH_OBJECT) || - find_cached_object(oid)) + hash_object_file(repo->hash_algo, buf, len, type, oid); + if (repo_has_object_file_with_flags(repo, oid, OBJECT_INFO_QUICK | OBJECT_INFO_SKIP_FETCH_OBJECT) || + find_cached_object(repo->objects, oid)) return 0; - ALLOC_GROW(cached_objects, cached_object_nr + 1, cached_object_alloc); - co = &cached_objects[cached_object_nr++]; + + ALLOC_GROW(repo->objects->cached_objects, + repo->objects->cached_object_nr + 1, repo->objects->cached_object_alloc); + co = &repo->objects->cached_objects[repo->objects->cached_object_nr++]; co->value.size = len; co->value.type = type; co_buf = xmalloc(len); @@ -1021,6 +1024,10 @@ void raw_object_store_clear(struct raw_object_store *o) o->odb_tail = NULL; o->loaded_alternates = 0; + for (size_t i = 0; i < o->cached_object_nr; i++) + free((char *) o->cached_objects[i].value.buf); + FREE_AND_NULL(o->cached_objects); + INIT_LIST_HEAD(&o->packed_git_mru); close_object_store(o); From patchwork Tue Apr 8 10:24:17 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Patrick Steinhardt X-Patchwork-Id: 14042671 Received: from fhigh-a1-smtp.messagingengine.com (fhigh-a1-smtp.messagingengine.com [103.168.172.152]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 6608C26F467 for ; Tue, 8 Apr 2025 10:24:23 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=103.168.172.152 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1744107868; cv=none; b=qoo2SIs1FDedqfywjZr3H3QQBkGkUGfixlph+8ISnJotR+VVi7rNF+6aHXzy3hxzrVARtppko5dbFfLiXHsnUyxwB1ERm3S9KWGCW6Mbdzwi7BcS+fy1fBpWDDJpif40UCPWEBi8xclmR9AwbUuwsh9tYzZV3OAw3NgAVvXf7Yk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1744107868; c=relaxed/simple; bh=jKRug5gliRHG4OnvdGwWpGpEghzyTnfNUN11AIzPxpw=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=GswufRTvlRLNxdzR+j76PwQoxzazF8wGTSQvZq1xvau7cmg8rVElfn4ZnkiVKhnMksaqdiRxamk9Ajpbv1CCpRDcmYxXFghDqY4j7gBkrdb4f0OhtGd6QARoQOrXqOYflz3SUOc6ca2sACcCJafJ8OzTMRcK+dWZJ8TAQbbLpFo= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=pks.im; spf=pass smtp.mailfrom=pks.im; dkim=pass (2048-bit key) header.d=pks.im header.i=@pks.im header.b=YI6Ta7/n; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b=eLPbLSsK; arc=none smtp.client-ip=103.168.172.152 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=pks.im Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=pks.im Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=pks.im header.i=@pks.im header.b="YI6Ta7/n"; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b="eLPbLSsK" Received: from phl-compute-05.internal (phl-compute-05.phl.internal [10.202.2.45]) by mailfhigh.phl.internal (Postfix) with ESMTP id 58C1A114011C for ; Tue, 8 Apr 2025 06:24:22 -0400 (EDT) Received: from phl-mailfrontend-01 ([10.202.2.162]) by phl-compute-05.internal (MEProxy); Tue, 08 Apr 2025 06:24:22 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=pks.im; h=cc:cc :content-transfer-encoding:content-type:content-type:date:date :from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:subject:subject:to:to; s=fm2; t=1744107862; x=1744194262; bh=wKkhTZ4IRe3gKJFvT8GFjaqOMpkuDd1vfd3GNc08UrE=; b= YI6Ta7/niQVEsNMROzpT9oM+EjQLZ1J45tt8RQrgnbJU9WiBeEdlHr/pOSYmv1O3 3cfkXNszbAaBNVmnnqioWVIbuzdvsXrS/iTxZvXnDP2Wl+JiuLo7tgOgGaTq4X4q 9cyvqgbyRXahvc3YIMXxDJVdasCRUsXseM0t2mrU5FGNnPk3aYC5pM87fnth1qmV ZFFgtE4UwDpOQsSYXvaD6YnaeYoTeFj6yuNfgRp4kJvHUFWSUpLptGxgrbEnUOVJ u55C8f1zqeHl/r96F4N3H0WJLKOnoAEH9fsL3SNw5c8H1M0DhvTfF0T519ffHWGf e99iUB/DKiKWKdjZjnY6Gw== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-transfer-encoding :content-type:content-type:date:date:feedback-id:feedback-id :from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:subject:subject:to:to:x-me-proxy :x-me-sender:x-me-sender:x-sasl-enc; s=fm2; t=1744107862; x= 1744194262; bh=wKkhTZ4IRe3gKJFvT8GFjaqOMpkuDd1vfd3GNc08UrE=; b=e LPbLSsKSz2omx4w+xqKa+R0z/67+2wmOSxcMiqTmB2WaWeJCVBJ25ctcThYNu/iF ytc4AxatVawCz6RPPwHcN8NlhRNGS2qqnPb0XhmyyutLWH+3naXv7lbKl4elY5j6 VeMQWhTDv1plWIRD07nYLns67yj1wQucmh55sgi40hIqdpS5LCM58iINq0V4NZ83 LkUveBsPrzLbtWicSZQst8ql53d05KXwAKArWzgAxYGY5W3wOQaJbQEQuNNqMCU4 Ha6QMRoa1Ee1SjpkxtGTs83+rgZqz+nd2hTOYLsUWACIfn0cepNiex+DHv1ojYFj VO0V9ZLebRulRSvTN3hng== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeefvddrtddtgddvtddvkeegucetufdoteggodetrf dotffvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdggtfgfnhhsuhgsshgtrhhisggv pdfurfetoffkrfgpnffqhgenuceurghilhhouhhtmecufedttdenucenucfjughrpefhff fugggtgffkfhgjvfevofesthejredtredtjeenucfhrhhomheprfgrthhrihgtkhcuufht vghinhhhrghrughtuceophhssehpkhhsrdhimheqnecuggftrfgrthhtvghrnhepffeuie dujedvkeehuedvkeefffeivdeuleetkeduheejteekgedvudfgtdfgieelnecuvehluhhs thgvrhfuihiivgeptdenucfrrghrrghmpehmrghilhhfrhhomhepphhssehpkhhsrdhimh dpnhgspghrtghpthhtohepuddpmhhouggvpehsmhhtphhouhhtpdhrtghpthhtohepghhi thesvhhgvghrrdhkvghrnhgvlhdrohhrgh X-ME-Proxy: Feedback-ID: i197146af:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA for ; Tue, 8 Apr 2025 06:24:21 -0400 (EDT) Received: by vm-mail (OpenSMTPD) with ESMTPSA id d32df9df (TLSv1.3:TLS_AES_256_GCM_SHA384:256:NO) for ; Tue, 8 Apr 2025 10:24:20 +0000 (UTC) From: Patrick Steinhardt Date: Tue, 08 Apr 2025 12:24:17 +0200 Subject: [PATCH 9/9] object-store: merge "object-store-ll.h" and "object-store.h" Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20250408-pks-split-object-file-v1-9-f1fd50191143@pks.im> References: <20250408-pks-split-object-file-v1-0-f1fd50191143@pks.im> In-Reply-To: <20250408-pks-split-object-file-v1-0-f1fd50191143@pks.im> To: git@vger.kernel.org Cc: X-Mailer: b4 0.14.2 The "object-store-ll.h" header has been introduced to keep transitive header dependendcies and compile times at bay. Now that we have created a new "object-store.c" file though we can easily move the last remaining additional bit of "object-store.h", the `odb_path_map`, out of the header. Do so. As the "object-store.h" header is now equivalent to its low-level alternative we drop the latter and inline it into the former. Signed-off-by: Patrick Steinhardt --- apply.c | 2 +- archive-tar.c | 2 +- archive-zip.c | 2 +- archive.c | 2 +- attr.c | 2 +- bisect.c | 2 +- blame.c | 2 +- builtin/backfill.c | 2 +- builtin/blame.c | 2 +- builtin/cat-file.c | 2 +- builtin/checkout.c | 2 +- builtin/clone.c | 2 +- builtin/commit-graph.c | 2 +- builtin/commit-tree.c | 2 +- builtin/count-objects.c | 2 +- builtin/describe.c | 2 +- builtin/difftool.c | 2 +- builtin/fast-export.c | 2 +- builtin/fast-import.c | 2 +- builtin/fetch.c | 2 +- builtin/fsck.c | 2 +- builtin/gc.c | 2 +- builtin/grep.c | 2 +- builtin/hash-object.c | 2 +- builtin/index-pack.c | 2 +- builtin/log.c | 2 +- builtin/ls-tree.c | 2 +- builtin/merge-tree.c | 2 +- builtin/mktag.c | 2 +- builtin/mktree.c | 2 +- builtin/multi-pack-index.c | 2 +- builtin/notes.c | 2 +- builtin/pack-objects.c | 2 +- builtin/pack-redundant.c | 2 +- builtin/prune.c | 2 +- builtin/receive-pack.c | 2 +- builtin/remote.c | 2 +- builtin/repack.c | 2 +- builtin/replace.c | 2 +- builtin/rev-list.c | 2 +- builtin/show-ref.c | 2 +- builtin/submodule--helper.c | 2 +- builtin/tag.c | 2 +- builtin/unpack-file.c | 2 +- builtin/unpack-objects.c | 2 +- bulk-checkin.c | 2 +- bundle-uri.c | 2 +- bundle.c | 2 +- cache-tree.c | 2 +- combine-diff.c | 2 +- commit-graph.c | 2 +- commit-graph.h | 2 +- commit.c | 2 +- config.c | 2 +- connected.c | 2 +- convert.c | 2 +- diagnose.c | 2 +- diff.c | 2 +- diffcore-rename.c | 2 +- dir.c | 2 +- entry.c | 2 +- fetch-pack.c | 2 +- fmt-merge-msg.c | 2 +- fsck.c | 2 +- grep.c | 2 +- http-backend.c | 2 +- http-push.c | 2 +- http-walker.c | 2 +- http.c | 2 +- list-objects-filter.c | 2 +- list-objects.c | 2 +- log-tree.c | 2 +- mailmap.c | 2 +- merge-blobs.c | 2 +- merge-recursive.c | 2 +- notes-merge.c | 2 +- object-file.c | 1 + object-name.c | 2 +- object-store-ll.h | 517 ------------------------------------------ object-store.c | 5 + object-store.h | 516 ++++++++++++++++++++++++++++++++++++++++- oss-fuzz/fuzz-pack-idx.c | 2 +- pack-bitmap-write.c | 2 +- pack-bitmap.c | 2 +- pack-check.c | 2 +- pack-mtimes.c | 2 +- pack-objects.h | 2 +- pack-revindex.c | 2 +- packfile.c | 2 +- path.c | 2 +- promisor-remote.c | 2 +- protocol-caps.c | 2 +- prune-packed.c | 2 +- reachable.c | 2 +- read-cache.c | 2 +- ref-filter.c | 2 +- reflog.c | 2 +- refs.c | 2 +- remote.c | 2 +- replace-object.c | 2 +- replace-object.h | 2 +- repository.c | 2 +- rerere.c | 2 +- revision.c | 2 +- send-pack.c | 2 +- sequencer.c | 2 +- server-info.c | 2 +- shallow.c | 2 +- streaming.c | 2 +- submodule-config.c | 2 +- submodule.c | 2 +- t/helper/test-pack-mtimes.c | 2 +- t/helper/test-partial-clone.c | 2 +- t/helper/test-read-graph.c | 2 +- t/helper/test-read-midx.c | 2 +- t/helper/test-ref-store.c | 2 +- tag.c | 2 +- tmp-objdir.c | 2 +- tree-walk.c | 2 +- tree.c | 2 +- unpack-trees.c | 2 +- upload-pack.c | 2 +- walker.c | 2 +- xdiff-interface.c | 2 +- 124 files changed, 637 insertions(+), 642 deletions(-) diff --git a/apply.c b/apply.c index f274a379487..2b6f4d0af87 100644 --- a/apply.c +++ b/apply.c @@ -14,7 +14,7 @@ #include "abspath.h" #include "base85.h" #include "config.h" -#include "object-store-ll.h" +#include "object-store.h" #include "delta.h" #include "diff.h" #include "dir.h" diff --git a/archive-tar.c b/archive-tar.c index 0edf13fba75..282b48196f9 100644 --- a/archive-tar.c +++ b/archive-tar.c @@ -11,7 +11,7 @@ #include "hex.h" #include "tar.h" #include "archive.h" -#include "object-store-ll.h" +#include "object-store.h" #include "strbuf.h" #include "streaming.h" #include "run-command.h" diff --git a/archive-zip.c b/archive-zip.c index 9f32730181b..405da6f3d83 100644 --- a/archive-zip.c +++ b/archive-zip.c @@ -12,7 +12,7 @@ #include "hex.h" #include "streaming.h" #include "utf8.h" -#include "object-store-ll.h" +#include "object-store.h" #include "strbuf.h" #include "userdiff.h" #include "write-or-die.h" diff --git a/archive.c b/archive.c index c95e3981524..014c312178c 100644 --- a/archive.c +++ b/archive.c @@ -14,7 +14,7 @@ #include "pretty.h" #include "setup.h" #include "refs.h" -#include "object-store-ll.h" +#include "object-store.h" #include "commit.h" #include "tree.h" #include "tree-walk.h" diff --git a/attr.c b/attr.c index 0bd2750528f..86b6109fc4e 100644 --- a/attr.c +++ b/attr.c @@ -22,7 +22,7 @@ #include "read-cache-ll.h" #include "refs.h" #include "revision.h" -#include "object-store-ll.h" +#include "object-store.h" #include "setup.h" #include "thread-utils.h" #include "tree-walk.h" diff --git a/bisect.c b/bisect.c index 269a98bf978..a327468c75b 100644 --- a/bisect.c +++ b/bisect.c @@ -20,7 +20,7 @@ #include "commit-slab.h" #include "commit-reach.h" #include "object-name.h" -#include "object-store-ll.h" +#include "object-store.h" #include "path.h" #include "dir.h" diff --git a/blame.c b/blame.c index b7c5bd692e6..57daa45e899 100644 --- a/blame.c +++ b/blame.c @@ -3,7 +3,7 @@ #include "git-compat-util.h" #include "refs.h" -#include "object-store-ll.h" +#include "object-store.h" #include "cache-tree.h" #include "mergesort.h" #include "commit.h" diff --git a/builtin/backfill.c b/builtin/backfill.c index 33e1ea2f84f..aaa104bc91d 100644 --- a/builtin/backfill.c +++ b/builtin/backfill.c @@ -13,7 +13,7 @@ #include "tree.h" #include "tree-walk.h" #include "object.h" -#include "object-store-ll.h" +#include "object-store.h" #include "oid-array.h" #include "oidset.h" #include "promisor-remote.h" diff --git a/builtin/blame.c b/builtin/blame.c index c470654c7ec..4e156bfd19d 100644 --- a/builtin/blame.c +++ b/builtin/blame.c @@ -28,7 +28,7 @@ #include "line-log.h" #include "progress.h" #include "object-name.h" -#include "object-store-ll.h" +#include "object-store.h" #include "pager.h" #include "blame.h" #include "refs.h" diff --git a/builtin/cat-file.c b/builtin/cat-file.c index b13561cf73b..c870fde260a 100644 --- a/builtin/cat-file.c +++ b/builtin/cat-file.c @@ -22,7 +22,7 @@ #include "packfile.h" #include "object-file.h" #include "object-name.h" -#include "object-store-ll.h" +#include "object-store.h" #include "replace-object.h" #include "promisor-remote.h" #include "mailmap.h" diff --git a/builtin/checkout.c b/builtin/checkout.c index 3e68623838a..8136962e2b3 100644 --- a/builtin/checkout.c +++ b/builtin/checkout.c @@ -20,7 +20,7 @@ #include "merge-recursive.h" #include "object-file.h" #include "object-name.h" -#include "object-store-ll.h" +#include "object-store.h" #include "parse-options.h" #include "path.h" #include "preload-index.h" diff --git a/builtin/clone.c b/builtin/clone.c index 2993acb630e..d0423a2198b 100644 --- a/builtin/clone.c +++ b/builtin/clone.c @@ -25,7 +25,7 @@ #include "refs.h" #include "refspec.h" #include "object-file.h" -#include "object-store-ll.h" +#include "object-store.h" #include "tree.h" #include "tree-walk.h" #include "unpack-trees.h" diff --git a/builtin/commit-graph.c b/builtin/commit-graph.c index 8ca75262c59..be06d0a811b 100644 --- a/builtin/commit-graph.c +++ b/builtin/commit-graph.c @@ -6,7 +6,7 @@ #include "hex.h" #include "parse-options.h" #include "commit-graph.h" -#include "object-store-ll.h" +#include "object-store.h" #include "progress.h" #include "replace-object.h" #include "strbuf.h" diff --git a/builtin/commit-tree.c b/builtin/commit-tree.c index 38457600a4e..6f9975e7a88 100644 --- a/builtin/commit-tree.c +++ b/builtin/commit-tree.c @@ -9,7 +9,7 @@ #include "gettext.h" #include "hex.h" #include "object-name.h" -#include "object-store-ll.h" +#include "object-store.h" #include "commit.h" #include "parse-options.h" diff --git a/builtin/count-objects.c b/builtin/count-objects.c index 1e89148ed74..0bb5360b2f2 100644 --- a/builtin/count-objects.c +++ b/builtin/count-objects.c @@ -12,7 +12,7 @@ #include "parse-options.h" #include "quote.h" #include "packfile.h" -#include "object-store-ll.h" +#include "object-store.h" static unsigned long garbage; static off_t size_garbage; diff --git a/builtin/describe.c b/builtin/describe.c index 23df333fd04..0f87fbceef3 100644 --- a/builtin/describe.c +++ b/builtin/describe.c @@ -19,7 +19,7 @@ #include "setup.h" #include "strvec.h" #include "run-command.h" -#include "object-store-ll.h" +#include "object-store.h" #include "list-objects.h" #include "commit-slab.h" #include "wildmatch.h" diff --git a/builtin/difftool.c b/builtin/difftool.c index 45d6ea8a801..e6e45436531 100644 --- a/builtin/difftool.c +++ b/builtin/difftool.c @@ -29,7 +29,7 @@ #include "strbuf.h" #include "lockfile.h" #include "object-file.h" -#include "object-store-ll.h" +#include "object-store.h" #include "dir.h" #include "entry.h" #include "setup.h" diff --git a/builtin/fast-export.c b/builtin/fast-export.c index 170126d41ac..afacd228b5d 100644 --- a/builtin/fast-export.c +++ b/builtin/fast-export.c @@ -14,7 +14,7 @@ #include "refs.h" #include "refspec.h" #include "object-file.h" -#include "object-store-ll.h" +#include "object-store.h" #include "commit.h" #include "object.h" #include "tag.h" diff --git a/builtin/fast-import.c b/builtin/fast-import.c index 63880b595cc..0357211ea6c 100644 --- a/builtin/fast-import.c +++ b/builtin/fast-import.c @@ -24,7 +24,7 @@ #include "packfile.h" #include "object-file.h" #include "object-name.h" -#include "object-store-ll.h" +#include "object-store.h" #include "mem-pool.h" #include "commit-reach.h" #include "khash.h" diff --git a/builtin/fetch.c b/builtin/fetch.c index 02af5054690..b52a32a5e0d 100644 --- a/builtin/fetch.c +++ b/builtin/fetch.c @@ -14,7 +14,7 @@ #include "refs.h" #include "refspec.h" #include "object-name.h" -#include "object-store-ll.h" +#include "object-store.h" #include "oidset.h" #include "oid-array.h" #include "commit.h" diff --git a/builtin/fsck.c b/builtin/fsck.c index 32d40d8f9fc..47055ebaf9a 100644 --- a/builtin/fsck.c +++ b/builtin/fsck.c @@ -18,7 +18,7 @@ #include "packfile.h" #include "object-file.h" #include "object-name.h" -#include "object-store-ll.h" +#include "object-store.h" #include "path.h" #include "read-cache-ll.h" #include "replace-object.h" diff --git a/builtin/gc.c b/builtin/gc.c index b069629676c..def922e7145 100644 --- a/builtin/gc.c +++ b/builtin/gc.c @@ -29,7 +29,7 @@ #include "commit.h" #include "commit-graph.h" #include "packfile.h" -#include "object-store-ll.h" +#include "object-store.h" #include "pack.h" #include "pack-objects.h" #include "path.h" diff --git a/builtin/grep.c b/builtin/grep.c index 283d64cab80..bcfbe5be5ba 100644 --- a/builtin/grep.c +++ b/builtin/grep.c @@ -26,7 +26,7 @@ #include "submodule-config.h" #include "object-file.h" #include "object-name.h" -#include "object-store-ll.h" +#include "object-store.h" #include "packfile.h" #include "pager.h" #include "path.h" diff --git a/builtin/hash-object.c b/builtin/hash-object.c index 7aa889b1b4a..e4eddd63bf4 100644 --- a/builtin/hash-object.c +++ b/builtin/hash-object.c @@ -11,7 +11,7 @@ #include "gettext.h" #include "hex.h" #include "object-file.h" -#include "object-store-ll.h" +#include "object-store.h" #include "blob.h" #include "quote.h" #include "parse-options.h" diff --git a/builtin/index-pack.c b/builtin/index-pack.c index de127c0ff13..60a8ee05dbc 100644 --- a/builtin/index-pack.c +++ b/builtin/index-pack.c @@ -21,7 +21,7 @@ #include "packfile.h" #include "pack-revindex.h" #include "object-file.h" -#include "object-store-ll.h" +#include "object-store.h" #include "oid-array.h" #include "oidset.h" #include "path.h" diff --git a/builtin/log.c b/builtin/log.c index 06ffaa93e86..72ac45e9433 100644 --- a/builtin/log.c +++ b/builtin/log.c @@ -16,7 +16,7 @@ #include "hex.h" #include "refs.h" #include "object-name.h" -#include "object-store-ll.h" +#include "object-store.h" #include "pager.h" #include "color.h" #include "commit.h" diff --git a/builtin/ls-tree.c b/builtin/ls-tree.c index 8542b5d53e4..8aafc30ca48 100644 --- a/builtin/ls-tree.c +++ b/builtin/ls-tree.c @@ -10,7 +10,7 @@ #include "gettext.h" #include "hex.h" #include "object-name.h" -#include "object-store-ll.h" +#include "object-store.h" #include "tree.h" #include "path.h" #include "quote.h" diff --git a/builtin/merge-tree.c b/builtin/merge-tree.c index 3ec7127b3a6..4aafa73c615 100644 --- a/builtin/merge-tree.c +++ b/builtin/merge-tree.c @@ -10,7 +10,7 @@ #include "commit-reach.h" #include "merge-ort.h" #include "object-name.h" -#include "object-store-ll.h" +#include "object-store.h" #include "parse-options.h" #include "blob.h" #include "merge-blobs.h" diff --git a/builtin/mktag.c b/builtin/mktag.c index 6e188dce50a..7ac11c46d53 100644 --- a/builtin/mktag.c +++ b/builtin/mktag.c @@ -6,7 +6,7 @@ #include "strbuf.h" #include "replace-object.h" #include "object-file.h" -#include "object-store-ll.h" +#include "object-store.h" #include "fsck.h" #include "config.h" diff --git a/builtin/mktree.c b/builtin/mktree.c index 0644f951161..7ffe6eefd8a 100644 --- a/builtin/mktree.c +++ b/builtin/mktree.c @@ -12,7 +12,7 @@ #include "tree.h" #include "parse-options.h" #include "object-file.h" -#include "object-store-ll.h" +#include "object-store.h" static struct treeent { unsigned mode; diff --git a/builtin/multi-pack-index.c b/builtin/multi-pack-index.c index 2a938466f53..d98410ca6c6 100644 --- a/builtin/multi-pack-index.c +++ b/builtin/multi-pack-index.c @@ -7,7 +7,7 @@ #include "midx.h" #include "strbuf.h" #include "trace2.h" -#include "object-store-ll.h" +#include "object-store.h" #include "replace-object.h" #include "repository.h" diff --git a/builtin/notes.c b/builtin/notes.c index 0dbc233752d..a3f433ca4c0 100644 --- a/builtin/notes.c +++ b/builtin/notes.c @@ -16,7 +16,7 @@ #include "notes.h" #include "object-file.h" #include "object-name.h" -#include "object-store-ll.h" +#include "object-store.h" #include "path.h" #include "pretty.h" diff --git a/builtin/pack-objects.c b/builtin/pack-objects.c index 163aab547fe..488c80f2cf3 100644 --- a/builtin/pack-objects.c +++ b/builtin/pack-objects.c @@ -32,7 +32,7 @@ #include "list.h" #include "packfile.h" #include "object-file.h" -#include "object-store-ll.h" +#include "object-store.h" #include "replace-object.h" #include "dir.h" #include "midx.h" diff --git a/builtin/pack-redundant.c b/builtin/pack-redundant.c index 3febe732f8e..5d1fc781761 100644 --- a/builtin/pack-redundant.c +++ b/builtin/pack-redundant.c @@ -13,7 +13,7 @@ #include "hex.h" #include "packfile.h" -#include "object-store-ll.h" +#include "object-store.h" #include "strbuf.h" #define BLKSIZE 512 diff --git a/builtin/prune.c b/builtin/prune.c index 8f52da8bd66..e930caa0c0a 100644 --- a/builtin/prune.c +++ b/builtin/prune.c @@ -17,7 +17,7 @@ #include "replace-object.h" #include "object-file.h" #include "object-name.h" -#include "object-store-ll.h" +#include "object-store.h" #include "shallow.h" static const char * const prune_usage[] = { diff --git a/builtin/receive-pack.c b/builtin/receive-pack.c index ee51bd76f60..be314879e82 100644 --- a/builtin/receive-pack.c +++ b/builtin/receive-pack.c @@ -33,7 +33,7 @@ #include "packfile.h" #include "object-file.h" #include "object-name.h" -#include "object-store-ll.h" +#include "object-store.h" #include "path.h" #include "protocol.h" #include "commit-reach.h" diff --git a/builtin/remote.c b/builtin/remote.c index 1b7aad88380..59481b3a82c 100644 --- a/builtin/remote.c +++ b/builtin/remote.c @@ -14,7 +14,7 @@ #include "rebase.h" #include "refs.h" #include "refspec.h" -#include "object-store-ll.h" +#include "object-store.h" #include "strvec.h" #include "commit-reach.h" #include "progress.h" diff --git a/builtin/repack.c b/builtin/repack.c index f3330ade7b8..1fd2874324a 100644 --- a/builtin/repack.c +++ b/builtin/repack.c @@ -17,7 +17,7 @@ #include "midx.h" #include "packfile.h" #include "prune-packed.h" -#include "object-store-ll.h" +#include "object-store.h" #include "promisor-remote.h" #include "shallow.h" #include "pack.h" diff --git a/builtin/replace.c b/builtin/replace.c index ce2948a9b2a..4204ca3605b 100644 --- a/builtin/replace.c +++ b/builtin/replace.c @@ -19,7 +19,7 @@ #include "run-command.h" #include "object-file.h" #include "object-name.h" -#include "object-store-ll.h" +#include "object-store.h" #include "read-cache.h" #include "replace-object.h" #include "tag.h" diff --git a/builtin/rev-list.c b/builtin/rev-list.c index bb26bee0d45..0170d79b62f 100644 --- a/builtin/rev-list.c +++ b/builtin/rev-list.c @@ -14,7 +14,7 @@ #include "object.h" #include "object-name.h" #include "object-file.h" -#include "object-store-ll.h" +#include "object-store.h" #include "pack-bitmap.h" #include "log-tree.h" #include "graph.h" diff --git a/builtin/show-ref.c b/builtin/show-ref.c index 285cd3e4338..f81209f23c3 100644 --- a/builtin/show-ref.c +++ b/builtin/show-ref.c @@ -5,7 +5,7 @@ #include "hex.h" #include "refs/refs-internal.h" #include "object-name.h" -#include "object-store-ll.h" +#include "object-store.h" #include "object.h" #include "string-list.h" #include "parse-options.h" diff --git a/builtin/submodule--helper.c b/builtin/submodule--helper.c index 570226ea166..a8d1200b7cb 100644 --- a/builtin/submodule--helper.c +++ b/builtin/submodule--helper.c @@ -28,7 +28,7 @@ #include "diff.h" #include "object-file.h" #include "object-name.h" -#include "object-store-ll.h" +#include "object-store.h" #include "advice.h" #include "branch.h" #include "list-objects-filter-options.h" diff --git a/builtin/tag.c b/builtin/tag.c index 536a01ff3ae..e6b372cebf5 100644 --- a/builtin/tag.c +++ b/builtin/tag.c @@ -19,7 +19,7 @@ #include "refs.h" #include "object-file.h" #include "object-name.h" -#include "object-store-ll.h" +#include "object-store.h" #include "path.h" #include "tag.h" #include "parse-options.h" diff --git a/builtin/unpack-file.c b/builtin/unpack-file.c index b19e5cabd03..e33acfc4ee4 100644 --- a/builtin/unpack-file.c +++ b/builtin/unpack-file.c @@ -4,7 +4,7 @@ #include "hex.h" #include "object-file.h" #include "object-name.h" -#include "object-store-ll.h" +#include "object-store.h" static char *create_temp_file(struct object_id *oid) { diff --git a/builtin/unpack-objects.c b/builtin/unpack-objects.c index 4078eab9252..661be789f13 100644 --- a/builtin/unpack-objects.c +++ b/builtin/unpack-objects.c @@ -9,7 +9,7 @@ #include "git-zlib.h" #include "hex.h" #include "object-file.h" -#include "object-store-ll.h" +#include "object-store.h" #include "object.h" #include "delta.h" #include "pack.h" diff --git a/bulk-checkin.c b/bulk-checkin.c index 0b9bad92868..b38937d739b 100644 --- a/bulk-checkin.c +++ b/bulk-checkin.c @@ -19,7 +19,7 @@ #include "tmp-objdir.h" #include "packfile.h" #include "object-file.h" -#include "object-store-ll.h" +#include "object-store.h" static int odb_transaction_nesting; diff --git a/bundle-uri.c b/bundle-uri.c index 744257c49c1..96d2ba726d9 100644 --- a/bundle-uri.c +++ b/bundle-uri.c @@ -14,7 +14,7 @@ #include "fetch-pack.h" #include "remote.h" #include "trace2.h" -#include "object-store-ll.h" +#include "object-store.h" static struct { enum bundle_list_heuristic heuristic; diff --git a/bundle.c b/bundle.c index d7ad6908433..d661c4ec214 100644 --- a/bundle.c +++ b/bundle.c @@ -7,7 +7,7 @@ #include "environment.h" #include "gettext.h" #include "hex.h" -#include "object-store-ll.h" +#include "object-store.h" #include "repository.h" #include "object.h" #include "commit.h" diff --git a/cache-tree.c b/cache-tree.c index 4c8167ea927..c0e1e9ee1d4 100644 --- a/cache-tree.c +++ b/cache-tree.c @@ -10,7 +10,7 @@ #include "cache-tree.h" #include "bulk-checkin.h" #include "object-file.h" -#include "object-store-ll.h" +#include "object-store.h" #include "read-cache-ll.h" #include "replace-object.h" #include "repository.h" diff --git a/combine-diff.c b/combine-diff.c index 553bf59fed6..dfae9f7995d 100644 --- a/combine-diff.c +++ b/combine-diff.c @@ -2,7 +2,7 @@ #define DISABLE_SIGN_COMPARE_WARNINGS #include "git-compat-util.h" -#include "object-store-ll.h" +#include "object-store.h" #include "commit.h" #include "convert.h" #include "diff.h" diff --git a/commit-graph.c b/commit-graph.c index 8060c358b84..a36c2f1e303 100644 --- a/commit-graph.c +++ b/commit-graph.c @@ -14,7 +14,7 @@ #include "refs.h" #include "hash-lookup.h" #include "commit-graph.h" -#include "object-store-ll.h" +#include "object-store.h" #include "oid-array.h" #include "path.h" #include "alloc.h" diff --git a/commit-graph.h b/commit-graph.h index 67819401954..13f662827d4 100644 --- a/commit-graph.h +++ b/commit-graph.h @@ -1,7 +1,7 @@ #ifndef COMMIT_GRAPH_H #define COMMIT_GRAPH_H -#include "object-store-ll.h" +#include "object-store.h" #include "oidset.h" #define GIT_TEST_COMMIT_GRAPH "GIT_TEST_COMMIT_GRAPH" diff --git a/commit.c b/commit.c index fbf4f8e87fd..00842678bd2 100644 --- a/commit.c +++ b/commit.c @@ -9,7 +9,7 @@ #include "hex.h" #include "repository.h" #include "object-name.h" -#include "object-store-ll.h" +#include "object-store.h" #include "utf8.h" #include "diff.h" #include "revision.h" diff --git a/config.c b/config.c index accb47e2d18..b18b5617fcd 100644 --- a/config.c +++ b/config.c @@ -31,7 +31,7 @@ #include "hashmap.h" #include "string-list.h" #include "object-name.h" -#include "object-store-ll.h" +#include "object-store.h" #include "pager.h" #include "path.h" #include "utf8.h" diff --git a/connected.c b/connected.c index 3099da84f33..4415388beba 100644 --- a/connected.c +++ b/connected.c @@ -3,7 +3,7 @@ #include "git-compat-util.h" #include "gettext.h" #include "hex.h" -#include "object-store-ll.h" +#include "object-store.h" #include "run-command.h" #include "sigchain.h" #include "connected.h" diff --git a/convert.c b/convert.c index 9cc0ca20ca0..8783e17941f 100644 --- a/convert.c +++ b/convert.c @@ -8,7 +8,7 @@ #include "copy.h" #include "gettext.h" #include "hex.h" -#include "object-store-ll.h" +#include "object-store.h" #include "attr.h" #include "run-command.h" #include "quote.h" diff --git a/diagnose.c b/diagnose.c index bd485effea2..b1be74be983 100644 --- a/diagnose.c +++ b/diagnose.c @@ -7,7 +7,7 @@ #include "gettext.h" #include "hex.h" #include "strvec.h" -#include "object-store-ll.h" +#include "object-store.h" #include "packfile.h" #include "parse-options.h" #include "repository.h" diff --git a/diff.c b/diff.c index c361199ab1b..9f2042fd1a5 100644 --- a/diff.c +++ b/diff.c @@ -23,7 +23,7 @@ #include "color.h" #include "run-command.h" #include "utf8.h" -#include "object-store-ll.h" +#include "object-store.h" #include "userdiff.h" #include "submodule.h" #include "hashmap.h" diff --git a/diffcore-rename.c b/diffcore-rename.c index 5002e896aad..787a2cef5f4 100644 --- a/diffcore-rename.c +++ b/diffcore-rename.c @@ -8,7 +8,7 @@ #include "git-compat-util.h" #include "diff.h" #include "diffcore.h" -#include "object-store-ll.h" +#include "object-store.h" #include "hashmap.h" #include "mem-pool.h" #include "oid-array.h" diff --git a/dir.c b/dir.c index 16ae3b5169d..1f8b6e2aa67 100644 --- a/dir.c +++ b/dir.c @@ -17,7 +17,7 @@ #include "environment.h" #include "gettext.h" #include "name-hash.h" -#include "object-store-ll.h" +#include "object-store.h" #include "path.h" #include "refs.h" #include "repository.h" diff --git a/entry.c b/entry.c index 81b321e53d1..f36ec5ad242 100644 --- a/entry.c +++ b/entry.c @@ -1,7 +1,7 @@ #define USE_THE_REPOSITORY_VARIABLE #include "git-compat-util.h" -#include "object-store-ll.h" +#include "object-store.h" #include "dir.h" #include "environment.h" #include "gettext.h" diff --git a/fetch-pack.c b/fetch-pack.c index 1ed5e11dd56..210dc30d50f 100644 --- a/fetch-pack.c +++ b/fetch-pack.c @@ -24,7 +24,7 @@ #include "oid-array.h" #include "oidset.h" #include "packfile.h" -#include "object-store-ll.h" +#include "object-store.h" #include "path.h" #include "connected.h" #include "fetch-negotiator.h" diff --git a/fmt-merge-msg.c b/fmt-merge-msg.c index 5b63c3b088a..501b5acdd44 100644 --- a/fmt-merge-msg.c +++ b/fmt-merge-msg.c @@ -6,7 +6,7 @@ #include "environment.h" #include "refs.h" #include "object-name.h" -#include "object-store-ll.h" +#include "object-store.h" #include "diff.h" #include "diff-merges.h" #include "hex.h" diff --git a/fsck.c b/fsck.c index 9fc4c25ffd5..8dc8472ceb3 100644 --- a/fsck.c +++ b/fsck.c @@ -4,7 +4,7 @@ #include "date.h" #include "dir.h" #include "hex.h" -#include "object-store-ll.h" +#include "object-store.h" #include "path.h" #include "repository.h" #include "object.h" diff --git a/grep.c b/grep.c index 9284b5741f5..f8d535182c3 100644 --- a/grep.c +++ b/grep.c @@ -5,7 +5,7 @@ #include "gettext.h" #include "grep.h" #include "hex.h" -#include "object-store-ll.h" +#include "object-store.h" #include "pretty.h" #include "userdiff.h" #include "xdiff-interface.h" diff --git a/http-backend.c b/http-backend.c index 50b2858fad6..0c575aa88aa 100644 --- a/http-backend.c +++ b/http-backend.c @@ -18,7 +18,7 @@ #include "url.h" #include "strvec.h" #include "packfile.h" -#include "object-store-ll.h" +#include "object-store.h" #include "protocol.h" #include "date.h" #include "write-or-die.h" diff --git a/http-push.c b/http-push.c index 806eb67cf1b..32e37565f4e 100644 --- a/http-push.c +++ b/http-push.c @@ -20,7 +20,7 @@ #include "url.h" #include "packfile.h" #include "object-file.h" -#include "object-store-ll.h" +#include "object-store.h" #include "commit-reach.h" #ifdef EXPAT_NEEDS_XMLPARSE_H diff --git a/http-walker.c b/http-walker.c index 7918ddc0968..882cae19c24 100644 --- a/http-walker.c +++ b/http-walker.c @@ -9,7 +9,7 @@ #include "list.h" #include "transport.h" #include "packfile.h" -#include "object-store-ll.h" +#include "object-store.h" struct alt_base { char *base; diff --git a/http.c b/http.c index 0c9a872809f..bc18ff83c4b 100644 --- a/http.c +++ b/http.c @@ -19,7 +19,7 @@ #include "packfile.h" #include "string-list.h" #include "object-file.h" -#include "object-store-ll.h" +#include "object-store.h" #include "tempfile.h" static struct trace_key trace_curl = TRACE_KEY_INIT(CURL); diff --git a/list-objects-filter.c b/list-objects-filter.c index dc598a081bb..7765761b3c6 100644 --- a/list-objects-filter.c +++ b/list-objects-filter.c @@ -12,7 +12,7 @@ #include "oidmap.h" #include "oidset.h" #include "object-name.h" -#include "object-store-ll.h" +#include "object-store.h" /* Remember to update object flag allocation in object.h */ /* diff --git a/list-objects.c b/list-objects.c index 943e62e868f..1e5512e1318 100644 --- a/list-objects.c +++ b/list-objects.c @@ -14,7 +14,7 @@ #include "list-objects-filter.h" #include "list-objects-filter-options.h" #include "packfile.h" -#include "object-store-ll.h" +#include "object-store.h" #include "trace.h" #include "environment.h" diff --git a/log-tree.c b/log-tree.c index 5dd1b63076f..a4d4ab59ca0 100644 --- a/log-tree.c +++ b/log-tree.c @@ -9,7 +9,7 @@ #include "environment.h" #include "hex.h" #include "object-name.h" -#include "object-store-ll.h" +#include "object-store.h" #include "repository.h" #include "tmp-objdir.h" #include "commit.h" diff --git a/mailmap.c b/mailmap.c index f35d20ed7fd..9e2642a043b 100644 --- a/mailmap.c +++ b/mailmap.c @@ -6,7 +6,7 @@ #include "string-list.h" #include "mailmap.h" #include "object-name.h" -#include "object-store-ll.h" +#include "object-store.h" #include "setup.h" char *git_mailmap_file; diff --git a/merge-blobs.c b/merge-blobs.c index 0ad0390fea5..53f36dbc175 100644 --- a/merge-blobs.c +++ b/merge-blobs.c @@ -4,7 +4,7 @@ #include "merge-ll.h" #include "blob.h" #include "merge-blobs.h" -#include "object-store-ll.h" +#include "object-store.h" static int fill_mmfile_blob(mmfile_t *f, struct blob *obj) { diff --git a/merge-recursive.c b/merge-recursive.c index 9aedffc546b..981e57698ed 100644 --- a/merge-recursive.c +++ b/merge-recursive.c @@ -27,7 +27,7 @@ #include "name-hash.h" #include "object-file.h" #include "object-name.h" -#include "object-store-ll.h" +#include "object-store.h" #include "path.h" #include "repository.h" #include "revision.h" diff --git a/notes-merge.c b/notes-merge.c index 13750490eff..96e59322a8b 100644 --- a/notes-merge.c +++ b/notes-merge.c @@ -8,7 +8,7 @@ #include "refs.h" #include "object-file.h" #include "object-name.h" -#include "object-store-ll.h" +#include "object-store.h" #include "path.h" #include "read-cache.h" #include "repository.h" diff --git a/object-file.c b/object-file.c index 235d29858ff..4abca6cb613 100644 --- a/object-file.c +++ b/object-file.c @@ -12,6 +12,7 @@ #include "git-compat-util.h" #include "bulk-checkin.h" +#include "dir.h" #include "environment.h" #include "gettext.h" #include "hex.h" diff --git a/object-name.c b/object-name.c index 91f731373a1..2c751a5352a 100644 --- a/object-name.c +++ b/object-name.c @@ -19,7 +19,7 @@ #include "oidtree.h" #include "packfile.h" #include "pretty.h" -#include "object-store-ll.h" +#include "object-store.h" #include "read-cache-ll.h" #include "repo-settings.h" #include "repository.h" diff --git a/object-store-ll.h b/object-store-ll.h deleted file mode 100644 index bb5e8798a1b..00000000000 --- a/object-store-ll.h +++ /dev/null @@ -1,517 +0,0 @@ -#ifndef OBJECT_STORE_LL_H -#define OBJECT_STORE_LL_H - -#include "hashmap.h" -#include "object.h" -#include "list.h" -#include "thread-utils.h" -#include "oidset.h" - -struct oidmap; -struct oidtree; -struct strbuf; -struct repository; - -struct object_directory { - struct object_directory *next; - - /* - * Used to store the results of readdir(3) calls when we are OK - * sacrificing accuracy due to races for speed. That includes - * object existence with OBJECT_INFO_QUICK, as well as - * our search for unique abbreviated hashes. Don't use it for tasks - * requiring greater accuracy! - * - * Be sure to call odb_load_loose_cache() before using. - */ - uint32_t loose_objects_subdir_seen[8]; /* 256 bits */ - struct oidtree *loose_objects_cache; - - /* Map between object IDs for loose objects. */ - struct loose_object_map *loose_map; - - /* - * This is a temporary object store created by the tmp_objdir - * facility. Disable ref updates since the objects in the store - * might be discarded on rollback. - */ - int disable_ref_updates; - - /* - * This object store is ephemeral, so there is no need to fsync. - */ - int will_destroy; - - /* - * Path to the alternative object store. If this is a relative path, - * it is relative to the current working directory. - */ - char *path; -}; - -void prepare_alt_odb(struct repository *r); -int has_alt_odb(struct repository *r); -char *compute_alternate_path(const char *path, struct strbuf *err); -struct object_directory *find_odb(struct repository *r, const char *obj_dir); -typedef int alt_odb_fn(struct object_directory *, void *); -int foreach_alt_odb(alt_odb_fn, void*); -typedef void alternate_ref_fn(const struct object_id *oid, void *); -void for_each_alternate_ref(alternate_ref_fn, void *); - -/* - * Add the directory to the on-disk alternates file; the new entry will also - * take effect in the current process. - */ -void add_to_alternates_file(const char *dir); - -/* - * Add the directory to the in-memory list of alternates (along with any - * recursive alternates it points to), but do not modify the on-disk alternates - * file. - */ -void add_to_alternates_memory(const char *dir); - -/* - * Replace the current writable object directory with the specified temporary - * object directory; returns the former primary object directory. - */ -struct object_directory *set_temporary_primary_odb(const char *dir, int will_destroy); - -/* - * Restore a previous ODB replaced by set_temporary_main_odb. - */ -void restore_primary_odb(struct object_directory *restore_odb, const char *old_path); - -/* - * Populate and return the loose object cache array corresponding to the - * given object ID. - */ -struct oidtree *odb_loose_cache(struct object_directory *odb, - const struct object_id *oid); - -/* Empty the loose object cache for the specified object directory. */ -void odb_clear_loose_cache(struct object_directory *odb); - -struct packed_git { - struct hashmap_entry packmap_ent; - struct packed_git *next; - struct list_head mru; - struct pack_window *windows; - off_t pack_size; - const void *index_data; - size_t index_size; - uint32_t num_objects; - size_t crc_offset; - struct oidset bad_objects; - int index_version; - time_t mtime; - int pack_fd; - int index; /* for builtin/pack-objects.c */ - unsigned pack_local:1, - pack_keep:1, - pack_keep_in_core:1, - freshened:1, - do_not_close:1, - pack_promisor:1, - multi_pack_index:1, - is_cruft:1; - unsigned char hash[GIT_MAX_RAWSZ]; - struct revindex_entry *revindex; - const uint32_t *revindex_data; - const uint32_t *revindex_map; - size_t revindex_size; - /* - * mtimes_map points at the beginning of the memory mapped region of - * this pack's corresponding .mtimes file, and mtimes_size is the size - * of that .mtimes file - */ - const uint32_t *mtimes_map; - size_t mtimes_size; - - /* repo denotes the repository this packfile belongs to */ - struct repository *repo; - - /* something like ".git/objects/pack/xxxxx.pack" */ - char pack_name[FLEX_ARRAY]; /* more */ -}; - -struct multi_pack_index; - -static inline int pack_map_entry_cmp(const void *cmp_data UNUSED, - const struct hashmap_entry *entry, - const struct hashmap_entry *entry2, - const void *keydata) -{ - const char *key = keydata; - const struct packed_git *pg1, *pg2; - - pg1 = container_of(entry, const struct packed_git, packmap_ent); - pg2 = container_of(entry2, const struct packed_git, packmap_ent); - - return strcmp(pg1->pack_name, key ? key : pg2->pack_name); -} - -struct cached_object_entry; - -struct raw_object_store { - /* - * Set of all object directories; the main directory is first (and - * cannot be NULL after initialization). Subsequent directories are - * alternates. - */ - struct object_directory *odb; - struct object_directory **odb_tail; - struct kh_odb_path_map *odb_by_path; - - int loaded_alternates; - - /* - * A list of alternate object directories loaded from the environment; - * this should not generally need to be accessed directly, but will - * populate the "odb" list when prepare_alt_odb() is run. - */ - char *alternate_db; - - /* - * Objects that should be substituted by other objects - * (see git-replace(1)). - */ - struct oidmap *replace_map; - unsigned replace_map_initialized : 1; - pthread_mutex_t replace_mutex; /* protect object replace functions */ - - struct commit_graph *commit_graph; - unsigned commit_graph_attempted : 1; /* if loading has been attempted */ - - /* - * private data - * - * should only be accessed directly by packfile.c and midx.c - */ - struct multi_pack_index *multi_pack_index; - - /* - * private data - * - * should only be accessed directly by packfile.c - */ - - struct packed_git *packed_git; - /* A most-recently-used ordered version of the packed_git list. */ - struct list_head packed_git_mru; - - struct { - struct packed_git **packs; - unsigned flags; - } kept_pack_cache; - - /* - * This is meant to hold a *small* number of objects that you would - * want repo_read_object_file() to be able to return, but yet you do not want - * to write them into the object store (e.g. a browse-only - * application). - */ - struct cached_object_entry *cached_objects; - size_t cached_object_nr, cached_object_alloc; - - /* - * A map of packfiles to packed_git structs for tracking which - * packs have been loaded already. - */ - struct hashmap pack_map; - - /* - * A fast, rough count of the number of objects in the repository. - * These two fields are not meant for direct access. Use - * repo_approximate_object_count() instead. - */ - unsigned long approximate_object_count; - unsigned approximate_object_count_valid : 1; - - /* - * Whether packed_git has already been populated with this repository's - * packs. - */ - unsigned packed_git_initialized : 1; -}; - -struct raw_object_store *raw_object_store_new(void); -void raw_object_store_clear(struct raw_object_store *o); - -/* - * Create a temporary file rooted in the object database directory, or - * die on failure. The filename is taken from "pattern", which should have the - * usual "XXXXXX" trailer, and the resulting filename is written into the - * "template" buffer. Returns the open descriptor. - */ -int odb_mkstemp(struct strbuf *temp_filename, const char *pattern); - -/* - * Create a pack .keep file named "name" (which should generally be the output - * of odb_pack_name). Returns a file descriptor opened for writing, or -1 on - * error. - */ -int odb_pack_keep(const char *name); - -/* - * Put in `buf` the name of the file in the local object database that - * would be used to store a loose object with the specified oid. - */ -const char *loose_object_path(struct repository *r, struct strbuf *buf, - const struct object_id *oid); - -void *map_loose_object(struct repository *r, const struct object_id *oid, - unsigned long *size); - -void *repo_read_object_file(struct repository *r, - const struct object_id *oid, - enum object_type *type, - unsigned long *size); - -/* Read and unpack an object file into memory, write memory to an object file */ -int oid_object_info(struct repository *r, const struct object_id *, unsigned long *); - -void hash_object_file(const struct git_hash_algo *algo, const void *buf, - unsigned long len, enum object_type type, - struct object_id *oid); - -/* - * Add an object file to the in-memory object store, without writing it - * to disk. - * - * Callers are responsible for calling write_object_file to record the - * object in persistent storage before writing any other new objects - * that reference it. - */ -int pretend_object_file(struct repository *repo, - void *buf, unsigned long len, enum object_type type, - struct object_id *oid); - -struct object_info { - /* Request */ - enum object_type *typep; - unsigned long *sizep; - off_t *disk_sizep; - struct object_id *delta_base_oid; - struct strbuf *type_name; - void **contentp; - - /* Response */ - enum { - OI_CACHED, - OI_LOOSE, - OI_PACKED, - OI_DBCACHED - } whence; - union { - /* - * struct { - * ... Nothing to expose in this case - * } cached; - * struct { - * ... Nothing to expose in this case - * } loose; - */ - struct { - struct packed_git *pack; - off_t offset; - unsigned int is_delta; - } packed; - } u; -}; - -/* - * Initializer for a "struct object_info" that wants no items. You may - * also memset() the memory to all-zeroes. - */ -#define OBJECT_INFO_INIT { 0 } - -/* Invoke lookup_replace_object() on the given hash */ -#define OBJECT_INFO_LOOKUP_REPLACE 1 -/* Allow reading from a loose object file of unknown/bogus type */ -#define OBJECT_INFO_ALLOW_UNKNOWN_TYPE 2 -/* Do not retry packed storage after checking packed and loose storage */ -#define OBJECT_INFO_QUICK 8 -/* - * Do not attempt to fetch the object if missing (even if fetch_is_missing is - * nonzero). - */ -#define OBJECT_INFO_SKIP_FETCH_OBJECT 16 -/* - * This is meant for bulk prefetching of missing blobs in a partial - * clone. Implies OBJECT_INFO_SKIP_FETCH_OBJECT and OBJECT_INFO_QUICK - */ -#define OBJECT_INFO_FOR_PREFETCH (OBJECT_INFO_SKIP_FETCH_OBJECT | OBJECT_INFO_QUICK) - -/* Die if object corruption (not just an object being missing) was detected. */ -#define OBJECT_INFO_DIE_IF_CORRUPT 32 - -int oid_object_info_extended(struct repository *r, - const struct object_id *, - struct object_info *, unsigned flags); - -/* Retry packed storage after checking packed and loose storage */ -#define HAS_OBJECT_RECHECK_PACKED 1 - -/* - * Returns 1 if the object exists. This function will not lazily fetch objects - * in a partial clone. - */ -int has_object(struct repository *r, const struct object_id *oid, - unsigned flags); - -/* - * These macros and functions are deprecated. If checking existence for an - * object that is likely to be missing and/or whose absence is relatively - * inconsequential (or is consequential but the caller is prepared to handle - * it), use has_object(), which has better defaults (no lazy fetch in a partial - * clone and no rechecking of packed storage). In the unlikely event that a - * caller needs to assert existence of an object that it fully expects to - * exist, and wants to trigger a lazy fetch in a partial clone, use - * oid_object_info_extended() with a NULL struct object_info. - * - * These functions can be removed once all callers have migrated to - * has_object() and/or oid_object_info_extended(). - */ -int repo_has_object_file(struct repository *r, const struct object_id *oid); -int repo_has_object_file_with_flags(struct repository *r, - const struct object_id *oid, int flags); - -void assert_oid_type(const struct object_id *oid, enum object_type expect); - -/* - * Enabling the object read lock allows multiple threads to safely call the - * following functions in parallel: repo_read_object_file(), - * read_object_with_reference(), oid_object_info() and oid_object_info_extended(). - * - * obj_read_lock() and obj_read_unlock() may also be used to protect other - * section which cannot execute in parallel with object reading. Since the used - * lock is a recursive mutex, these sections can even contain calls to object - * reading functions. However, beware that in these cases zlib inflation won't - * be performed in parallel, losing performance. - * - * TODO: oid_object_info_extended()'s call stack has a recursive behavior. If - * any of its callees end up calling it, this recursive call won't benefit from - * parallel inflation. - */ -void enable_obj_read_lock(void); -void disable_obj_read_lock(void); - -extern int obj_read_use_lock; -extern pthread_mutex_t obj_read_mutex; - -static inline void obj_read_lock(void) -{ - if(obj_read_use_lock) - pthread_mutex_lock(&obj_read_mutex); -} - -static inline void obj_read_unlock(void) -{ - if(obj_read_use_lock) - pthread_mutex_unlock(&obj_read_mutex); -} - -/* - * Iterate over the files in the loose-object parts of the object - * directory "path", triggering the following callbacks: - * - * - loose_object is called for each loose object we find. - * - * - loose_cruft is called for any files that do not appear to be - * loose objects. Note that we only look in the loose object - * directories "objects/[0-9a-f]{2}/", so we will not report - * "objects/foobar" as cruft. - * - * - loose_subdir is called for each top-level hashed subdirectory - * of the object directory (e.g., "$OBJDIR/f0"). It is called - * after the objects in the directory are processed. - * - * Any callback that is NULL will be ignored. Callbacks returning non-zero - * will end the iteration. - * - * In the "buf" variant, "path" is a strbuf which will also be used as a - * scratch buffer, but restored to its original contents before - * the function returns. - */ -typedef int each_loose_object_fn(const struct object_id *oid, - const char *path, - void *data); -typedef int each_loose_cruft_fn(const char *basename, - const char *path, - void *data); -typedef int each_loose_subdir_fn(unsigned int nr, - const char *path, - void *data); -int for_each_file_in_obj_subdir(unsigned int subdir_nr, - struct strbuf *path, - each_loose_object_fn obj_cb, - each_loose_cruft_fn cruft_cb, - each_loose_subdir_fn subdir_cb, - void *data); -int for_each_loose_file_in_objdir(const char *path, - each_loose_object_fn obj_cb, - each_loose_cruft_fn cruft_cb, - each_loose_subdir_fn subdir_cb, - void *data); -int for_each_loose_file_in_objdir_buf(struct strbuf *path, - each_loose_object_fn obj_cb, - each_loose_cruft_fn cruft_cb, - each_loose_subdir_fn subdir_cb, - void *data); - -/* Flags for for_each_*_object() below. */ -enum for_each_object_flags { - /* Iterate only over local objects, not alternates. */ - FOR_EACH_OBJECT_LOCAL_ONLY = (1<<0), - - /* Only iterate over packs obtained from the promisor remote. */ - FOR_EACH_OBJECT_PROMISOR_ONLY = (1<<1), - - /* - * Visit objects within a pack in packfile order rather than .idx order - */ - FOR_EACH_OBJECT_PACK_ORDER = (1<<2), - - /* Only iterate over packs that are not marked as kept in-core. */ - FOR_EACH_OBJECT_SKIP_IN_CORE_KEPT_PACKS = (1<<3), - - /* Only iterate over packs that do not have .keep files. */ - FOR_EACH_OBJECT_SKIP_ON_DISK_KEPT_PACKS = (1<<4), -}; - -/* - * Iterate over all accessible loose objects without respect to - * reachability. By default, this includes both local and alternate objects. - * The order in which objects are visited is unspecified. - * - * Any flags specific to packs are ignored. - */ -int for_each_loose_object(each_loose_object_fn, void *, - enum for_each_object_flags flags); - -/* - * Iterate over all accessible packed objects without respect to reachability. - * By default, this includes both local and alternate packs. - * - * Note that some objects may appear twice if they are found in multiple packs. - * Each pack is visited in an unspecified order. By default, objects within a - * pack are visited in pack-idx order (i.e., sorted by oid). - */ -typedef int each_packed_object_fn(const struct object_id *oid, - struct packed_git *pack, - uint32_t pos, - void *data); -int for_each_object_in_pack(struct packed_git *p, - each_packed_object_fn, void *data, - enum for_each_object_flags flags); -int for_each_packed_object(struct repository *repo, each_packed_object_fn cb, - void *data, enum for_each_object_flags flags); - -void *read_object_with_reference(struct repository *r, - const struct object_id *oid, - enum object_type required_type, - unsigned long *size, - struct object_id *oid_ret); - -#endif /* OBJECT_STORE_LL_H */ diff --git a/object-store.c b/object-store.c index 17fa06a86fa..fc48925d230 100644 --- a/object-store.c +++ b/object-store.c @@ -4,9 +4,11 @@ #include "abspath.h" #include "commit-graph.h" #include "config.h" +#include "dir.h" #include "environment.h" #include "gettext.h" #include "hex.h" +#include "khash.h" #include "lockfile.h" #include "loose.h" #include "object-file-convert.h" @@ -24,6 +26,9 @@ #include "submodule.h" #include "write-or-die.h" +KHASH_INIT(odb_path_map, const char * /* key: odb_path */, + struct object_directory *, 1, fspathhash, fspatheq) + /* * This is meant to hold a *small* number of objects that you would * want repo_read_object_file() to be able to return, but yet you do not want diff --git a/object-store.h b/object-store.h index 1b3e3d7d014..46961dc9542 100644 --- a/object-store.h +++ b/object-store.h @@ -1,11 +1,517 @@ #ifndef OBJECT_STORE_H #define OBJECT_STORE_H -#include "khash.h" -#include "dir.h" -#include "object-store-ll.h" +#include "hashmap.h" +#include "object.h" +#include "list.h" +#include "oidset.h" +#include "thread-utils.h" -KHASH_INIT(odb_path_map, const char * /* key: odb_path */, - struct object_directory *, 1, fspathhash, fspatheq) +struct oidmap; +struct oidtree; +struct strbuf; +struct repository; + +struct object_directory { + struct object_directory *next; + + /* + * Used to store the results of readdir(3) calls when we are OK + * sacrificing accuracy due to races for speed. That includes + * object existence with OBJECT_INFO_QUICK, as well as + * our search for unique abbreviated hashes. Don't use it for tasks + * requiring greater accuracy! + * + * Be sure to call odb_load_loose_cache() before using. + */ + uint32_t loose_objects_subdir_seen[8]; /* 256 bits */ + struct oidtree *loose_objects_cache; + + /* Map between object IDs for loose objects. */ + struct loose_object_map *loose_map; + + /* + * This is a temporary object store created by the tmp_objdir + * facility. Disable ref updates since the objects in the store + * might be discarded on rollback. + */ + int disable_ref_updates; + + /* + * This object store is ephemeral, so there is no need to fsync. + */ + int will_destroy; + + /* + * Path to the alternative object store. If this is a relative path, + * it is relative to the current working directory. + */ + char *path; +}; + +void prepare_alt_odb(struct repository *r); +int has_alt_odb(struct repository *r); +char *compute_alternate_path(const char *path, struct strbuf *err); +struct object_directory *find_odb(struct repository *r, const char *obj_dir); +typedef int alt_odb_fn(struct object_directory *, void *); +int foreach_alt_odb(alt_odb_fn, void*); +typedef void alternate_ref_fn(const struct object_id *oid, void *); +void for_each_alternate_ref(alternate_ref_fn, void *); + +/* + * Add the directory to the on-disk alternates file; the new entry will also + * take effect in the current process. + */ +void add_to_alternates_file(const char *dir); + +/* + * Add the directory to the in-memory list of alternates (along with any + * recursive alternates it points to), but do not modify the on-disk alternates + * file. + */ +void add_to_alternates_memory(const char *dir); + +/* + * Replace the current writable object directory with the specified temporary + * object directory; returns the former primary object directory. + */ +struct object_directory *set_temporary_primary_odb(const char *dir, int will_destroy); + +/* + * Restore a previous ODB replaced by set_temporary_main_odb. + */ +void restore_primary_odb(struct object_directory *restore_odb, const char *old_path); + +/* + * Populate and return the loose object cache array corresponding to the + * given object ID. + */ +struct oidtree *odb_loose_cache(struct object_directory *odb, + const struct object_id *oid); + +/* Empty the loose object cache for the specified object directory. */ +void odb_clear_loose_cache(struct object_directory *odb); + +struct packed_git { + struct hashmap_entry packmap_ent; + struct packed_git *next; + struct list_head mru; + struct pack_window *windows; + off_t pack_size; + const void *index_data; + size_t index_size; + uint32_t num_objects; + size_t crc_offset; + struct oidset bad_objects; + int index_version; + time_t mtime; + int pack_fd; + int index; /* for builtin/pack-objects.c */ + unsigned pack_local:1, + pack_keep:1, + pack_keep_in_core:1, + freshened:1, + do_not_close:1, + pack_promisor:1, + multi_pack_index:1, + is_cruft:1; + unsigned char hash[GIT_MAX_RAWSZ]; + struct revindex_entry *revindex; + const uint32_t *revindex_data; + const uint32_t *revindex_map; + size_t revindex_size; + /* + * mtimes_map points at the beginning of the memory mapped region of + * this pack's corresponding .mtimes file, and mtimes_size is the size + * of that .mtimes file + */ + const uint32_t *mtimes_map; + size_t mtimes_size; + + /* repo denotes the repository this packfile belongs to */ + struct repository *repo; + + /* something like ".git/objects/pack/xxxxx.pack" */ + char pack_name[FLEX_ARRAY]; /* more */ +}; + +struct multi_pack_index; + +static inline int pack_map_entry_cmp(const void *cmp_data UNUSED, + const struct hashmap_entry *entry, + const struct hashmap_entry *entry2, + const void *keydata) +{ + const char *key = keydata; + const struct packed_git *pg1, *pg2; + + pg1 = container_of(entry, const struct packed_git, packmap_ent); + pg2 = container_of(entry2, const struct packed_git, packmap_ent); + + return strcmp(pg1->pack_name, key ? key : pg2->pack_name); +} + +struct cached_object_entry; + +struct raw_object_store { + /* + * Set of all object directories; the main directory is first (and + * cannot be NULL after initialization). Subsequent directories are + * alternates. + */ + struct object_directory *odb; + struct object_directory **odb_tail; + struct kh_odb_path_map *odb_by_path; + + int loaded_alternates; + + /* + * A list of alternate object directories loaded from the environment; + * this should not generally need to be accessed directly, but will + * populate the "odb" list when prepare_alt_odb() is run. + */ + char *alternate_db; + + /* + * Objects that should be substituted by other objects + * (see git-replace(1)). + */ + struct oidmap *replace_map; + unsigned replace_map_initialized : 1; + pthread_mutex_t replace_mutex; /* protect object replace functions */ + + struct commit_graph *commit_graph; + unsigned commit_graph_attempted : 1; /* if loading has been attempted */ + + /* + * private data + * + * should only be accessed directly by packfile.c and midx.c + */ + struct multi_pack_index *multi_pack_index; + + /* + * private data + * + * should only be accessed directly by packfile.c + */ + + struct packed_git *packed_git; + /* A most-recently-used ordered version of the packed_git list. */ + struct list_head packed_git_mru; + + struct { + struct packed_git **packs; + unsigned flags; + } kept_pack_cache; + + /* + * This is meant to hold a *small* number of objects that you would + * want repo_read_object_file() to be able to return, but yet you do not want + * to write them into the object store (e.g. a browse-only + * application). + */ + struct cached_object_entry *cached_objects; + size_t cached_object_nr, cached_object_alloc; + + /* + * A map of packfiles to packed_git structs for tracking which + * packs have been loaded already. + */ + struct hashmap pack_map; + + /* + * A fast, rough count of the number of objects in the repository. + * These two fields are not meant for direct access. Use + * repo_approximate_object_count() instead. + */ + unsigned long approximate_object_count; + unsigned approximate_object_count_valid : 1; + + /* + * Whether packed_git has already been populated with this repository's + * packs. + */ + unsigned packed_git_initialized : 1; +}; + +struct raw_object_store *raw_object_store_new(void); +void raw_object_store_clear(struct raw_object_store *o); + +/* + * Create a temporary file rooted in the object database directory, or + * die on failure. The filename is taken from "pattern", which should have the + * usual "XXXXXX" trailer, and the resulting filename is written into the + * "template" buffer. Returns the open descriptor. + */ +int odb_mkstemp(struct strbuf *temp_filename, const char *pattern); + +/* + * Create a pack .keep file named "name" (which should generally be the output + * of odb_pack_name). Returns a file descriptor opened for writing, or -1 on + * error. + */ +int odb_pack_keep(const char *name); + +/* + * Put in `buf` the name of the file in the local object database that + * would be used to store a loose object with the specified oid. + */ +const char *loose_object_path(struct repository *r, struct strbuf *buf, + const struct object_id *oid); + +void *map_loose_object(struct repository *r, const struct object_id *oid, + unsigned long *size); + +void *repo_read_object_file(struct repository *r, + const struct object_id *oid, + enum object_type *type, + unsigned long *size); + +/* Read and unpack an object file into memory, write memory to an object file */ +int oid_object_info(struct repository *r, const struct object_id *, unsigned long *); + +void hash_object_file(const struct git_hash_algo *algo, const void *buf, + unsigned long len, enum object_type type, + struct object_id *oid); + +/* + * Add an object file to the in-memory object store, without writing it + * to disk. + * + * Callers are responsible for calling write_object_file to record the + * object in persistent storage before writing any other new objects + * that reference it. + */ +int pretend_object_file(struct repository *repo, + void *buf, unsigned long len, enum object_type type, + struct object_id *oid); + +struct object_info { + /* Request */ + enum object_type *typep; + unsigned long *sizep; + off_t *disk_sizep; + struct object_id *delta_base_oid; + struct strbuf *type_name; + void **contentp; + + /* Response */ + enum { + OI_CACHED, + OI_LOOSE, + OI_PACKED, + OI_DBCACHED + } whence; + union { + /* + * struct { + * ... Nothing to expose in this case + * } cached; + * struct { + * ... Nothing to expose in this case + * } loose; + */ + struct { + struct packed_git *pack; + off_t offset; + unsigned int is_delta; + } packed; + } u; +}; + +/* + * Initializer for a "struct object_info" that wants no items. You may + * also memset() the memory to all-zeroes. + */ +#define OBJECT_INFO_INIT { 0 } + +/* Invoke lookup_replace_object() on the given hash */ +#define OBJECT_INFO_LOOKUP_REPLACE 1 +/* Allow reading from a loose object file of unknown/bogus type */ +#define OBJECT_INFO_ALLOW_UNKNOWN_TYPE 2 +/* Do not retry packed storage after checking packed and loose storage */ +#define OBJECT_INFO_QUICK 8 +/* + * Do not attempt to fetch the object if missing (even if fetch_is_missing is + * nonzero). + */ +#define OBJECT_INFO_SKIP_FETCH_OBJECT 16 +/* + * This is meant for bulk prefetching of missing blobs in a partial + * clone. Implies OBJECT_INFO_SKIP_FETCH_OBJECT and OBJECT_INFO_QUICK + */ +#define OBJECT_INFO_FOR_PREFETCH (OBJECT_INFO_SKIP_FETCH_OBJECT | OBJECT_INFO_QUICK) + +/* Die if object corruption (not just an object being missing) was detected. */ +#define OBJECT_INFO_DIE_IF_CORRUPT 32 + +int oid_object_info_extended(struct repository *r, + const struct object_id *, + struct object_info *, unsigned flags); + +/* Retry packed storage after checking packed and loose storage */ +#define HAS_OBJECT_RECHECK_PACKED 1 + +/* + * Returns 1 if the object exists. This function will not lazily fetch objects + * in a partial clone. + */ +int has_object(struct repository *r, const struct object_id *oid, + unsigned flags); + +/* + * These macros and functions are deprecated. If checking existence for an + * object that is likely to be missing and/or whose absence is relatively + * inconsequential (or is consequential but the caller is prepared to handle + * it), use has_object(), which has better defaults (no lazy fetch in a partial + * clone and no rechecking of packed storage). In the unlikely event that a + * caller needs to assert existence of an object that it fully expects to + * exist, and wants to trigger a lazy fetch in a partial clone, use + * oid_object_info_extended() with a NULL struct object_info. + * + * These functions can be removed once all callers have migrated to + * has_object() and/or oid_object_info_extended(). + */ +int repo_has_object_file(struct repository *r, const struct object_id *oid); +int repo_has_object_file_with_flags(struct repository *r, + const struct object_id *oid, int flags); + +void assert_oid_type(const struct object_id *oid, enum object_type expect); + +/* + * Enabling the object read lock allows multiple threads to safely call the + * following functions in parallel: repo_read_object_file(), + * read_object_with_reference(), oid_object_info() and oid_object_info_extended(). + * + * obj_read_lock() and obj_read_unlock() may also be used to protect other + * section which cannot execute in parallel with object reading. Since the used + * lock is a recursive mutex, these sections can even contain calls to object + * reading functions. However, beware that in these cases zlib inflation won't + * be performed in parallel, losing performance. + * + * TODO: oid_object_info_extended()'s call stack has a recursive behavior. If + * any of its callees end up calling it, this recursive call won't benefit from + * parallel inflation. + */ +void enable_obj_read_lock(void); +void disable_obj_read_lock(void); + +extern int obj_read_use_lock; +extern pthread_mutex_t obj_read_mutex; + +static inline void obj_read_lock(void) +{ + if(obj_read_use_lock) + pthread_mutex_lock(&obj_read_mutex); +} + +static inline void obj_read_unlock(void) +{ + if(obj_read_use_lock) + pthread_mutex_unlock(&obj_read_mutex); +} + +/* + * Iterate over the files in the loose-object parts of the object + * directory "path", triggering the following callbacks: + * + * - loose_object is called for each loose object we find. + * + * - loose_cruft is called for any files that do not appear to be + * loose objects. Note that we only look in the loose object + * directories "objects/[0-9a-f]{2}/", so we will not report + * "objects/foobar" as cruft. + * + * - loose_subdir is called for each top-level hashed subdirectory + * of the object directory (e.g., "$OBJDIR/f0"). It is called + * after the objects in the directory are processed. + * + * Any callback that is NULL will be ignored. Callbacks returning non-zero + * will end the iteration. + * + * In the "buf" variant, "path" is a strbuf which will also be used as a + * scratch buffer, but restored to its original contents before + * the function returns. + */ +typedef int each_loose_object_fn(const struct object_id *oid, + const char *path, + void *data); +typedef int each_loose_cruft_fn(const char *basename, + const char *path, + void *data); +typedef int each_loose_subdir_fn(unsigned int nr, + const char *path, + void *data); +int for_each_file_in_obj_subdir(unsigned int subdir_nr, + struct strbuf *path, + each_loose_object_fn obj_cb, + each_loose_cruft_fn cruft_cb, + each_loose_subdir_fn subdir_cb, + void *data); +int for_each_loose_file_in_objdir(const char *path, + each_loose_object_fn obj_cb, + each_loose_cruft_fn cruft_cb, + each_loose_subdir_fn subdir_cb, + void *data); +int for_each_loose_file_in_objdir_buf(struct strbuf *path, + each_loose_object_fn obj_cb, + each_loose_cruft_fn cruft_cb, + each_loose_subdir_fn subdir_cb, + void *data); + +/* Flags for for_each_*_object() below. */ +enum for_each_object_flags { + /* Iterate only over local objects, not alternates. */ + FOR_EACH_OBJECT_LOCAL_ONLY = (1<<0), + + /* Only iterate over packs obtained from the promisor remote. */ + FOR_EACH_OBJECT_PROMISOR_ONLY = (1<<1), + + /* + * Visit objects within a pack in packfile order rather than .idx order + */ + FOR_EACH_OBJECT_PACK_ORDER = (1<<2), + + /* Only iterate over packs that are not marked as kept in-core. */ + FOR_EACH_OBJECT_SKIP_IN_CORE_KEPT_PACKS = (1<<3), + + /* Only iterate over packs that do not have .keep files. */ + FOR_EACH_OBJECT_SKIP_ON_DISK_KEPT_PACKS = (1<<4), +}; + +/* + * Iterate over all accessible loose objects without respect to + * reachability. By default, this includes both local and alternate objects. + * The order in which objects are visited is unspecified. + * + * Any flags specific to packs are ignored. + */ +int for_each_loose_object(each_loose_object_fn, void *, + enum for_each_object_flags flags); + +/* + * Iterate over all accessible packed objects without respect to reachability. + * By default, this includes both local and alternate packs. + * + * Note that some objects may appear twice if they are found in multiple packs. + * Each pack is visited in an unspecified order. By default, objects within a + * pack are visited in pack-idx order (i.e., sorted by oid). + */ +typedef int each_packed_object_fn(const struct object_id *oid, + struct packed_git *pack, + uint32_t pos, + void *data); +int for_each_object_in_pack(struct packed_git *p, + each_packed_object_fn, void *data, + enum for_each_object_flags flags); +int for_each_packed_object(struct repository *repo, each_packed_object_fn cb, + void *data, enum for_each_object_flags flags); + +void *read_object_with_reference(struct repository *r, + const struct object_id *oid, + enum object_type required_type, + unsigned long *size, + struct object_id *oid_ret); #endif /* OBJECT_STORE_H */ diff --git a/oss-fuzz/fuzz-pack-idx.c b/oss-fuzz/fuzz-pack-idx.c index 3e190214d14..609a343ee3e 100644 --- a/oss-fuzz/fuzz-pack-idx.c +++ b/oss-fuzz/fuzz-pack-idx.c @@ -1,5 +1,5 @@ #include "git-compat-util.h" -#include "object-store-ll.h" +#include "object-store.h" #include "packfile.h" int LLVMFuzzerTestOneInput(const uint8_t *data, size_t size); diff --git a/pack-bitmap-write.c b/pack-bitmap-write.c index 6a97b52b36d..4c7f7985e78 100644 --- a/pack-bitmap-write.c +++ b/pack-bitmap-write.c @@ -4,7 +4,7 @@ #include "environment.h" #include "gettext.h" #include "hex.h" -#include "object-store-ll.h" +#include "object-store.h" #include "commit.h" #include "diff.h" #include "revision.h" diff --git a/pack-bitmap.c b/pack-bitmap.c index 0dbd7c4ffe1..fb68d2ae637 100644 --- a/pack-bitmap.c +++ b/pack-bitmap.c @@ -17,7 +17,7 @@ #include "packfile.h" #include "repository.h" #include "trace2.h" -#include "object-store-ll.h" +#include "object-store.h" #include "list-objects-filter-options.h" #include "midx.h" #include "config.h" diff --git a/pack-check.c b/pack-check.c index 95dcbbe9852..874897d6cba 100644 --- a/pack-check.c +++ b/pack-check.c @@ -8,7 +8,7 @@ #include "progress.h" #include "packfile.h" #include "object-file.h" -#include "object-store-ll.h" +#include "object-store.h" struct idx_entry { off_t offset; diff --git a/pack-mtimes.c b/pack-mtimes.c index bcea28e521d..20900ca88d3 100644 --- a/pack-mtimes.c +++ b/pack-mtimes.c @@ -1,7 +1,7 @@ #include "git-compat-util.h" #include "gettext.h" #include "pack-mtimes.h" -#include "object-store-ll.h" +#include "object-store.h" #include "packfile.h" #include "strbuf.h" diff --git a/pack-objects.h b/pack-objects.h index d73e3843c92..d1c4ae7f9b6 100644 --- a/pack-objects.h +++ b/pack-objects.h @@ -1,7 +1,7 @@ #ifndef PACK_OBJECTS_H #define PACK_OBJECTS_H -#include "object-store-ll.h" +#include "object-store.h" #include "thread-utils.h" #include "pack.h" diff --git a/pack-revindex.c b/pack-revindex.c index 1ee7b49e206..37ad9c79268 100644 --- a/pack-revindex.c +++ b/pack-revindex.c @@ -1,7 +1,7 @@ #include "git-compat-util.h" #include "gettext.h" #include "pack-revindex.h" -#include "object-store-ll.h" +#include "object-store.h" #include "packfile.h" #include "strbuf.h" #include "trace2.h" diff --git a/packfile.c b/packfile.c index 9d09f8bc726..d91016f1c7f 100644 --- a/packfile.c +++ b/packfile.c @@ -19,7 +19,7 @@ #include "tree-walk.h" #include "tree.h" #include "object-file.h" -#include "object-store-ll.h" +#include "object-store.h" #include "midx.h" #include "commit-graph.h" #include "pack-revindex.h" diff --git a/path.c b/path.c index 910756c8b32..1f7a014322c 100644 --- a/path.c +++ b/path.c @@ -15,7 +15,7 @@ #include "submodule-config.h" #include "path.h" #include "packfile.h" -#include "object-store-ll.h" +#include "object-store.h" #include "lockfile.h" #include "exec-cmd.h" diff --git a/promisor-remote.c b/promisor-remote.c index 5801ebfd9b2..9d058586dfa 100644 --- a/promisor-remote.c +++ b/promisor-remote.c @@ -3,7 +3,7 @@ #include "git-compat-util.h" #include "gettext.h" #include "hex.h" -#include "object-store-ll.h" +#include "object-store.h" #include "promisor-remote.h" #include "config.h" #include "trace2.h" diff --git a/protocol-caps.c b/protocol-caps.c index 855f279c2f7..9b8db37a210 100644 --- a/protocol-caps.c +++ b/protocol-caps.c @@ -6,7 +6,7 @@ #include "hash.h" #include "hex.h" #include "object.h" -#include "object-store-ll.h" +#include "object-store.h" #include "repository.h" #include "string-list.h" #include "strbuf.h" diff --git a/prune-packed.c b/prune-packed.c index 7dad2fc0c16..c1d95a519d7 100644 --- a/prune-packed.c +++ b/prune-packed.c @@ -2,7 +2,7 @@ #include "git-compat-util.h" #include "gettext.h" -#include "object-store-ll.h" +#include "object-store.h" #include "packfile.h" #include "progress.h" #include "prune-packed.h" diff --git a/reachable.c b/reachable.c index 1b26b9b1d76..16e23a38037 100644 --- a/reachable.c +++ b/reachable.c @@ -14,7 +14,7 @@ #include "list-objects.h" #include "packfile.h" #include "worktree.h" -#include "object-store-ll.h" +#include "object-store.h" #include "pack-bitmap.h" #include "pack-mtimes.h" #include "config.h" diff --git a/read-cache.c b/read-cache.c index 9909b56902f..9aef59cab4c 100644 --- a/read-cache.c +++ b/read-cache.c @@ -22,7 +22,7 @@ #include "refs.h" #include "dir.h" #include "object-file.h" -#include "object-store-ll.h" +#include "object-store.h" #include "oid-array.h" #include "tree.h" #include "commit.h" diff --git a/ref-filter.c b/ref-filter.c index 6da8d4c03b6..7a274633cfc 100644 --- a/ref-filter.c +++ b/ref-filter.c @@ -12,7 +12,7 @@ #include "refs.h" #include "wildmatch.h" #include "object-name.h" -#include "object-store-ll.h" +#include "object-store.h" #include "oid-array.h" #include "repo-settings.h" #include "repository.h" diff --git a/reflog.c b/reflog.c index 1b5f031f6d7..60834a124d9 100644 --- a/reflog.c +++ b/reflog.c @@ -3,7 +3,7 @@ #include "git-compat-util.h" #include "gettext.h" -#include "object-store-ll.h" +#include "object-store.h" #include "reflog.h" #include "refs.h" #include "revision.h" diff --git a/refs.c b/refs.c index 1208f86629b..14f25c499cd 100644 --- a/refs.c +++ b/refs.c @@ -19,7 +19,7 @@ #include "run-command.h" #include "hook.h" #include "object-name.h" -#include "object-store-ll.h" +#include "object-store.h" #include "object.h" #include "path.h" #include "submodule.h" diff --git a/remote.c b/remote.c index e609cf5c56a..7fc657d98b8 100644 --- a/remote.c +++ b/remote.c @@ -12,7 +12,7 @@ #include "refs.h" #include "refspec.h" #include "object-name.h" -#include "object-store-ll.h" +#include "object-store.h" #include "path.h" #include "commit.h" #include "diff.h" diff --git a/replace-object.c b/replace-object.c index 9a3cdd809a9..7b8a09b5cb4 100644 --- a/replace-object.c +++ b/replace-object.c @@ -2,7 +2,7 @@ #include "gettext.h" #include "hex.h" #include "oidmap.h" -#include "object-store-ll.h" +#include "object-store.h" #include "replace-object.h" #include "refs.h" #include "repository.h" diff --git a/replace-object.h b/replace-object.h index 66c41b938b4..ba478eb30c4 100644 --- a/replace-object.h +++ b/replace-object.h @@ -3,7 +3,7 @@ #include "oidmap.h" #include "repository.h" -#include "object-store-ll.h" +#include "object-store.h" struct replace_object { struct oidmap_entry original; diff --git a/repository.c b/repository.c index 6cbaf2e3daa..9b3d6665fc6 100644 --- a/repository.c +++ b/repository.c @@ -1,7 +1,7 @@ #include "git-compat-util.h" #include "abspath.h" #include "repository.h" -#include "object-store-ll.h" +#include "object-store.h" #include "config.h" #include "object.h" #include "lockfile.h" diff --git a/rerere.c b/rerere.c index 740e8ad1a0b..28e59c42380 100644 --- a/rerere.c +++ b/rerere.c @@ -18,7 +18,7 @@ #include "path.h" #include "pathspec.h" #include "object-file.h" -#include "object-store-ll.h" +#include "object-store.h" #include "strmap.h" #define RESOLVED 0 diff --git a/revision.c b/revision.c index b536c4a29ad..352e18b1f97 100644 --- a/revision.c +++ b/revision.c @@ -8,7 +8,7 @@ #include "hex.h" #include "object-name.h" #include "object-file.h" -#include "object-store-ll.h" +#include "object-store.h" #include "oidset.h" #include "tag.h" #include "blob.h" diff --git a/send-pack.c b/send-pack.c index 856a65d5f5a..5005689cb55 100644 --- a/send-pack.c +++ b/send-pack.c @@ -4,7 +4,7 @@ #include "date.h" #include "gettext.h" #include "hex.h" -#include "object-store-ll.h" +#include "object-store.h" #include "pkt-line.h" #include "sideband.h" #include "run-command.h" diff --git a/sequencer.c b/sequencer.c index c112d2e1c43..925cfa1af68 100644 --- a/sequencer.c +++ b/sequencer.c @@ -13,7 +13,7 @@ #include "dir.h" #include "object-file.h" #include "object-name.h" -#include "object-store-ll.h" +#include "object-store.h" #include "object.h" #include "pager.h" #include "commit.h" diff --git a/server-info.c b/server-info.c index 1ca0e00d51e..3048e5ee407 100644 --- a/server-info.c +++ b/server-info.c @@ -11,7 +11,7 @@ #include "packfile.h" #include "path.h" #include "object-file.h" -#include "object-store-ll.h" +#include "object-store.h" #include "server-info.h" #include "strbuf.h" #include "tempfile.h" diff --git a/shallow.c b/shallow.c index 06c3266a3e0..2f82ebd6e3f 100644 --- a/shallow.c +++ b/shallow.c @@ -5,7 +5,7 @@ #include "repository.h" #include "tempfile.h" #include "lockfile.h" -#include "object-store-ll.h" +#include "object-store.h" #include "commit.h" #include "tag.h" #include "pkt-line.h" diff --git a/streaming.c b/streaming.c index 018b794d252..127d6b5d6ac 100644 --- a/streaming.c +++ b/streaming.c @@ -10,7 +10,7 @@ #include "streaming.h" #include "repository.h" #include "object-file.h" -#include "object-store-ll.h" +#include "object-store.h" #include "replace-object.h" #include "packfile.h" diff --git a/submodule-config.c b/submodule-config.c index d82b404b73e..8630e27947d 100644 --- a/submodule-config.c +++ b/submodule-config.c @@ -13,7 +13,7 @@ #include "submodule.h" #include "strbuf.h" #include "object-name.h" -#include "object-store-ll.h" +#include "object-store.h" #include "parse-options.h" #include "thread-utils.h" #include "tree-walk.h" diff --git a/submodule.c b/submodule.c index 0821507ecaa..3e52ef3b036 100644 --- a/submodule.c +++ b/submodule.c @@ -27,7 +27,7 @@ #include "parse-options.h" #include "object-file.h" #include "object-name.h" -#include "object-store-ll.h" +#include "object-store.h" #include "commit-reach.h" #include "read-cache-ll.h" #include "setup.h" diff --git a/t/helper/test-pack-mtimes.c b/t/helper/test-pack-mtimes.c index f8f9afbb5b1..50f5941bff7 100644 --- a/t/helper/test-pack-mtimes.c +++ b/t/helper/test-pack-mtimes.c @@ -3,7 +3,7 @@ #include "test-tool.h" #include "hex.h" #include "strbuf.h" -#include "object-store-ll.h" +#include "object-store.h" #include "packfile.h" #include "pack-mtimes.h" #include "setup.h" diff --git a/t/helper/test-partial-clone.c b/t/helper/test-partial-clone.c index a1af9710c31..34f1aee5581 100644 --- a/t/helper/test-partial-clone.c +++ b/t/helper/test-partial-clone.c @@ -1,7 +1,7 @@ #include "test-tool.h" #include "hex.h" #include "repository.h" -#include "object-store-ll.h" +#include "object-store.h" #include "setup.h" /* diff --git a/t/helper/test-read-graph.c b/t/helper/test-read-graph.c index 811dde1cb3c..8b413b644be 100644 --- a/t/helper/test-read-graph.c +++ b/t/helper/test-read-graph.c @@ -3,7 +3,7 @@ #include "test-tool.h" #include "commit-graph.h" #include "repository.h" -#include "object-store-ll.h" +#include "object-store.h" #include "bloom.h" #include "setup.h" diff --git a/t/helper/test-read-midx.c b/t/helper/test-read-midx.c index fc632369618..ac81390899a 100644 --- a/t/helper/test-read-midx.c +++ b/t/helper/test-read-midx.c @@ -4,7 +4,7 @@ #include "hex.h" #include "midx.h" #include "repository.h" -#include "object-store-ll.h" +#include "object-store.h" #include "pack-bitmap.h" #include "packfile.h" #include "setup.h" diff --git a/t/helper/test-ref-store.c b/t/helper/test-ref-store.c index 2ff67c067ac..4cfc7c90b59 100644 --- a/t/helper/test-ref-store.c +++ b/t/helper/test-ref-store.c @@ -5,7 +5,7 @@ #include "refs.h" #include "setup.h" #include "worktree.h" -#include "object-store-ll.h" +#include "object-store.h" #include "path.h" #include "repository.h" #include "strbuf.h" diff --git a/tag.c b/tag.c index 8d9e9e29304..05be39067cf 100644 --- a/tag.c +++ b/tag.c @@ -5,7 +5,7 @@ #include "environment.h" #include "tag.h" #include "object-name.h" -#include "object-store-ll.h" +#include "object-store.h" #include "commit.h" #include "tree.h" #include "blob.h" diff --git a/tmp-objdir.c b/tmp-objdir.c index 31d16a4c2c5..c38fbeb5e8a 100644 --- a/tmp-objdir.c +++ b/tmp-objdir.c @@ -10,7 +10,7 @@ #include "strbuf.h" #include "strvec.h" #include "quote.h" -#include "object-store-ll.h" +#include "object-store.h" #include "repository.h" struct tmp_objdir { diff --git a/tree-walk.c b/tree-walk.c index a0333979656..90655d52378 100644 --- a/tree-walk.c +++ b/tree-walk.c @@ -6,7 +6,7 @@ #include "gettext.h" #include "hex.h" #include "object-file.h" -#include "object-store-ll.h" +#include "object-store.h" #include "trace2.h" #include "tree.h" #include "pathspec.h" diff --git a/tree.c b/tree.c index ad86ad1ba99..b85f56267fb 100644 --- a/tree.c +++ b/tree.c @@ -4,7 +4,7 @@ #include "hex.h" #include "tree.h" #include "object-name.h" -#include "object-store-ll.h" +#include "object-store.h" #include "commit.h" #include "alloc.h" #include "tree-walk.h" diff --git a/unpack-trees.c b/unpack-trees.c index cf5b73c84be..471837f0329 100644 --- a/unpack-trees.c +++ b/unpack-trees.c @@ -26,7 +26,7 @@ #include "symlinks.h" #include "trace2.h" #include "fsmonitor.h" -#include "object-store-ll.h" +#include "object-store.h" #include "promisor-remote.h" #include "entry.h" #include "parallel-checkout.h" diff --git a/upload-pack.c b/upload-pack.c index 02ce6336028..30e4630f3a1 100644 --- a/upload-pack.c +++ b/upload-pack.c @@ -10,7 +10,7 @@ #include "pkt-line.h" #include "sideband.h" #include "repository.h" -#include "object-store-ll.h" +#include "object-store.h" #include "oid-array.h" #include "object.h" #include "commit.h" diff --git a/walker.c b/walker.c index 1cf3da02193..4fedc19f346 100644 --- a/walker.c +++ b/walker.c @@ -5,7 +5,7 @@ #include "hex.h" #include "walker.h" #include "repository.h" -#include "object-store-ll.h" +#include "object-store.h" #include "commit.h" #include "strbuf.h" #include "tree.h" diff --git a/xdiff-interface.c b/xdiff-interface.c index 77712811ff1..1edcd319e6e 100644 --- a/xdiff-interface.c +++ b/xdiff-interface.c @@ -5,7 +5,7 @@ #include "gettext.h" #include "config.h" #include "hex.h" -#include "object-store-ll.h" +#include "object-store.h" #include "strbuf.h" #include "xdiff-interface.h" #include "xdiff/xtypes.h"