From patchwork Wed Jan 13 22:23:31 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Taylor Blau X-Patchwork-Id: 12018271 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3C028C433E0 for ; Thu, 14 Jan 2021 02:05:58 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id DDFBC235E4 for ; Thu, 14 Jan 2021 02:05:57 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727479AbhAMW2t (ORCPT ); Wed, 13 Jan 2021 17:28:49 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:55028 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1729302AbhAMWYv (ORCPT ); Wed, 13 Jan 2021 17:24:51 -0500 Received: from mail-qt1-x82e.google.com (mail-qt1-x82e.google.com [IPv6:2607:f8b0:4864:20::82e]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 073ABC061794 for ; Wed, 13 Jan 2021 14:23:35 -0800 (PST) Received: by mail-qt1-x82e.google.com with SMTP id g24so2285728qtq.12 for ; Wed, 13 Jan 2021 14:23:34 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ttaylorr-com.20150623.gappssmtp.com; s=20150623; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to; bh=r2Ij7b3enD/CrytVWJjzjfBbDMQQjC8jGACJJMCAAQI=; b=NO7CihYa1+Hwl8KxBon6MXQTKMHsYqW1q/jwe+henaz4dA6DmgHJUiHOPojndZGyOZ hv+29bycngQCgmXwtntrCoYQAHhXhPT7VSoM2P6gD66ZHvmNKLU8yXSgQ7sms8+lVlIb Q+IOyfsPlw583ILOdHMefB7hCeSPoCEkye31oa0kEBDvrpn7AfG27Ymlp4nSHAyiU6DZ WmCxvIsBGoQZ+42u8YadghgyclXyO2KndGHHNgGHpjrbe8pfeuFuCuymEmbEJ+Mzq7Y2 gcttsdVyCLWXpchDm56tl4lbnDgbpOm+IAV18/l0tWelNpeqMYuV7W6YaiC7kLQUUE/m cLVQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to; bh=r2Ij7b3enD/CrytVWJjzjfBbDMQQjC8jGACJJMCAAQI=; b=WXAHB3RzQlTlN4hT2zKm4orCV1Fq70Dz/dG6LgzhGNY/+sYHXloJ2wLugu9jDZ38Tr vL01Ewm7ZhxxOuhpIROaLZ0Klp2iOIbJEW8xPs3ECdo/CDbHQVoSyQ1cpwQy6qhPFW6h 0jIEoG/a9bsB7SNZpSFhqgg1e3KBy80L/WO1jtG4/k4YrgEi9Qjii72uR5gNPfEUjciM qovFV6OWwL+VV+z3RdWjHKaoh5YGydl5W/j2KP5R5PFAikTEoz2I3cVMQHAmwR3Q7mef axlK1lolPnezNgGqvdQveruCp/SXnXHys/hiuzS7799WVtQqkxfEi7UewwBJJCJREu7t Hqhg== X-Gm-Message-State: AOAM533ps6bOID/FlicKrk1kSrhLA9VUkkDy+OlBOerRTEPiIgSZDcsT W4DQNPLtQBJiOs9CF0Z6wTYblXOp1Kld1A== X-Google-Smtp-Source: ABdhPJxI1awlAZKxzivC6Rz0sQ6BaASAh+f41Q/g6fybVyt19W2htuO3PowoJHSmSur3++jlYDyVcw== X-Received: by 2002:ac8:128c:: with SMTP id y12mr4503266qti.127.1610576613830; Wed, 13 Jan 2021 14:23:33 -0800 (PST) Received: from localhost ([2605:9480:22e:ff10:b172:2e4c:efe4:db53]) by smtp.gmail.com with ESMTPSA id n4sm1860946qtl.22.2021.01.13.14.23.33 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 13 Jan 2021 14:23:33 -0800 (PST) Date: Wed, 13 Jan 2021 17:23:31 -0500 From: Taylor Blau To: git@vger.kernel.org Cc: dstolee@microsoft.com, gitster@pobox.com, jrnieder@gmail.com, peff@peff.net Subject: [PATCH v2 01/20] pack-revindex: introduce a new API Message-ID: References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org In the next several patches, we will prepare for loading a reverse index either in memory (mapping the inverse of the .idx's contents in-core), or directly from a yet-to-be-introduced on-disk format. To prepare for that, we'll introduce an API that avoids the caller explicitly indexing the revindex pointer in the packed_git structure. There are four ways to interact with the reverse index. Accordingly, four functions will be exported from 'pack-revindex.h' by the time that the existing API is removed. A caller may: 1. Load the pack's reverse index. This involves opening up the index, generating an array, and then sorting it. Since opening the index can fail, this function ('load_pack_revindex()') returns an int. Accordingly, it takes only a single argument: the 'struct packed_git' the caller wants to build a reverse index for. This function is well-suited for both the current and new API. Callers will have to continue to open the reverse index explicitly, but this function will eventually learn how to detect and load a reverse index from the on-disk format, if one exists. Otherwise, it will fallback to generating one in memory from scratch. 2. Convert a pack position into an offset. This operation is now called `pack_pos_to_offset()`. It takes a pack and a position, and returns the corresponding off_t. Any error simply calls BUG(), since the callers are not well-suited to handle a failure and keep going. 3. Convert a pack position into an index position. Same as above; this takes a pack and a position, and returns a uint32_t. This operation is known as `pack_pos_to_index()`. The same thinking about error conditions applies here as well. 4. Find the pack position for a given offset. This operation is now known as `offset_to_pack_pos()`. It takes a pack, an offset, and a pointer to a uint32_t where the position is written, if an object exists at that offset. Otherwise, -1 is returned to indicate failure. Unlike some of the callers that used to access '->offset' and '->nr' directly, the error checking around this call is somewhat more robust. This is important since callers should always pass an offset which points at the boundary of two objects. The API, unlike direct access, enforces that that is the case. This will become important in a subsequent patch where a caller which does not but could check the return value treats the signed `-1` from `find_revindex_position()` as an index into the 'revindex' array. Two design warts are carried over into the new API: - Asking for the index position of an out-of-bounds object will result in a BUG() (since no such object exists), but asking for the offset of the non-existent object at the end of the pack returns the total size of the pack. This makes it convenient for callers who always want to take the difference of two adjacent object's offsets (to compute the on-disk size) but don't want to worry about boundaries at the end of the pack. - offset_to_pack_pos() lazily loads the reverse index, but pack_pos_to_index() doesn't (callers of the former are well-suited to handle errors, but callers of the latter are not). Signed-off-by: Taylor Blau --- pack-revindex.c | 32 +++++++++++++++++++++++++++++ pack-revindex.h | 54 +++++++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 86 insertions(+) diff --git a/pack-revindex.c b/pack-revindex.c index ecdde39cf4..0ca3b54b45 100644 --- a/pack-revindex.c +++ b/pack-revindex.c @@ -203,3 +203,35 @@ struct revindex_entry *find_pack_revindex(struct packed_git *p, off_t ofs) return p->revindex + pos; } + +int offset_to_pack_pos(struct packed_git *p, off_t ofs, uint32_t *pos) +{ + int ret; + + if (load_pack_revindex(p) < 0) + return -1; + + ret = find_revindex_position(p, ofs); + if (ret < 0) + return ret; + *pos = ret; + return 0; +} + +uint32_t pack_pos_to_index(struct packed_git *p, uint32_t pos) +{ + if (!p->revindex) + BUG("pack_pos_to_index: reverse index not yet loaded"); + if (p->num_objects <= pos) + BUG("pack_pos_to_index: out-of-bounds object at %"PRIu32, pos); + return p->revindex[pos].nr; +} + +off_t pack_pos_to_offset(struct packed_git *p, uint32_t pos) +{ + if (!p->revindex) + BUG("pack_pos_to_index: reverse index not yet loaded"); + if (p->num_objects < pos) + BUG("pack_pos_to_offset: out-of-bounds object at %"PRIu32, pos); + return p->revindex[pos].offset; +} diff --git a/pack-revindex.h b/pack-revindex.h index 848331d5d6..5a218aaa66 100644 --- a/pack-revindex.h +++ b/pack-revindex.h @@ -1,6 +1,21 @@ #ifndef PACK_REVINDEX_H #define PACK_REVINDEX_H +/** + * A revindex allows converting efficiently between three properties + * of an object within a pack: + * + * - index position: the numeric position within the list of sorted object ids + * found in the .idx file + * + * - pack position: the numeric position within the list of objects in their + * order within the actual .pack file (i.e., 0 is the first object in the + * .pack, 1 is the second, and so on) + * + * - offset: the byte offset within the .pack file at which the object contents + * can be found + */ + struct packed_git; struct revindex_entry { @@ -8,9 +23,48 @@ struct revindex_entry { unsigned int nr; }; +/* + * load_pack_revindex populates the revindex's internal data-structures for the + * given pack, returning zero on success and a negative value otherwise. + */ int load_pack_revindex(struct packed_git *p); int find_revindex_position(struct packed_git *p, off_t ofs); struct revindex_entry *find_pack_revindex(struct packed_git *p, off_t ofs); +/* + * offset_to_pack_pos converts an object offset to a pack position. This + * function returns zero on success, and a negative number otherwise. The + * parameter 'pos' is usable only on success. + * + * If the reverse index has not yet been loaded, this function loads it lazily, + * and returns an negative number if an error was encountered. + * + * This function runs in time O(log N) with the number of objects in the pack. + */ +int offset_to_pack_pos(struct packed_git *p, off_t ofs, uint32_t *pos); + +/* + * pack_pos_to_index converts the given pack-relative position 'pos' by + * returning an index-relative position. + * + * If the reverse index has not yet been loaded, or the position is out of + * bounds, this function aborts. + * + * This function runs in constant time. + */ +uint32_t pack_pos_to_index(struct packed_git *p, uint32_t pos); + +/* + * pack_pos_to_offset converts the given pack-relative position 'pos' into a + * pack offset. For a pack with 'N' objects, asking for position 'N' will return + * the total size (in bytes) of the pack. + * + * If the reverse index has not yet been loaded, or the position is out of + * bounds, this function aborts. + * + * This function runs in constant time. + */ +off_t pack_pos_to_offset(struct packed_git *p, uint32_t pos); + #endif From patchwork Wed Jan 13 22:23:35 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Taylor Blau X-Patchwork-Id: 12018329 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5DC41C433DB for ; Thu, 14 Jan 2021 02:10:47 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 14A1D2242A for ; Thu, 14 Jan 2021 02:10:47 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1730194AbhANCJ7 (ORCPT ); Wed, 13 Jan 2021 21:09:59 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:55044 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1729299AbhAMWYv (ORCPT ); Wed, 13 Jan 2021 17:24:51 -0500 Received: from mail-qk1-x733.google.com (mail-qk1-x733.google.com [IPv6:2607:f8b0:4864:20::733]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 124DBC061795 for ; Wed, 13 Jan 2021 14:23:39 -0800 (PST) Received: by mail-qk1-x733.google.com with SMTP id w79so4434174qkb.5 for ; Wed, 13 Jan 2021 14:23:39 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ttaylorr-com.20150623.gappssmtp.com; s=20150623; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to; bh=FcrMgWKoL9JcvtnnDPFkP5fIWQ2mPWD0Tf8r1DdjpfA=; b=oyNBQzeutsAOaTKaqC7Hrvf2hh4hKNMZR3hJ7NAXbN7HpZckJ4c59aWNrSjnqkWG66 gpVx20Mqz/Iyxx3OKN1Ia+o5pIUet5ikGAhzyrnY9mzxDkPfJEWu+/8BNTx03Py3J73s 4uh1ctAk4ykxZ9IMMEvJuj5sQeaVXr1zi6d/lX9jfRbHMTTA57nRaxz2a9QCUMUU19Bn WvcGn9uBIx91LinFU1ai0Zy5Et08k/hv/LuiRnga2yPklvy4qVL6nPVZTF+xWyoNUb/1 pkZ9FfI9MRM1u04Pz+9Pf15gl5RhHmnnM0bwwY4iTGzwiYm4a7OmIhMtsqmItmUl3TgR XYWw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to; bh=FcrMgWKoL9JcvtnnDPFkP5fIWQ2mPWD0Tf8r1DdjpfA=; b=ino+Q6srzIQHOYqHPggQp7cJNmW4HncI0n5PEYLA5PezBw/p/3LCRK1Teer8MpaZRf a2MUiRHz6UHcEdnQ/MtngkYtcdzA3uHlOl0670ZnJSqAXdPvGiCO+DdOHtNs6qgw1czg 6fyxCdt88bUJYG60LfKfZb8ykoknUTkpSc/lx+FIPXJWkp+mmsl5zumTJPgU5QSoMULH RCyKoOYui5Q8KugCSuxAc662tu8SK3gDwVvcioQOnhErq0af2FvyOEx8rFwxEvHWEa3A tbo61+D/JN0bG/+UhdE1Gwy/m7KCWUsk9vtl/YWM+NQZ9JRfiPe4e33z8oMiRDDigaEy fdrw== X-Gm-Message-State: AOAM530XsrymS0BCRQf5GG4GlhR4M9hB3TQwFHWF3tp1IYJsm2F958yl faSYfLJygM4RsC4tV8F2OsauhCRCG0mUag== X-Google-Smtp-Source: ABdhPJy+H4Nkxx9z4m4iyG5RgX0XQPfqYZnd+4ybqG3XliOCuaYG2A8FMLDGBoel8oD6Unx7tHh4Dw== X-Received: by 2002:a37:65ca:: with SMTP id z193mr4353912qkb.171.1610576618104; Wed, 13 Jan 2021 14:23:38 -0800 (PST) Received: from localhost ([2605:9480:22e:ff10:b172:2e4c:efe4:db53]) by smtp.gmail.com with ESMTPSA id n195sm1890190qke.20.2021.01.13.14.23.37 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 13 Jan 2021 14:23:37 -0800 (PST) Date: Wed, 13 Jan 2021 17:23:35 -0500 From: Taylor Blau To: git@vger.kernel.org Cc: dstolee@microsoft.com, gitster@pobox.com, jrnieder@gmail.com, peff@peff.net Subject: [PATCH v2 02/20] write_reuse_object(): convert to new revindex API Message-ID: <0fca7d5812185d482fd48f7df6c062ab44933055.1610576604.git.me@ttaylorr.com> References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org First replace 'find_pack_revindex()' with its replacement 'offset_to_pack_pos()'. This prevents any bogus OFS_DELTA that may make its way through until 'write_reuse_object()' from causing a bad memory read (if 'revidx' is 'NULL') Next, replace a direct access of '->nr' with the wrapper function 'pack_pos_to_index()'. Signed-off-by: Taylor Blau --- builtin/pack-objects.c | 13 +++++++++---- 1 file changed, 9 insertions(+), 4 deletions(-) diff --git a/builtin/pack-objects.c b/builtin/pack-objects.c index 2a00358f34..ab1fd853f1 100644 --- a/builtin/pack-objects.c +++ b/builtin/pack-objects.c @@ -419,7 +419,7 @@ static off_t write_reuse_object(struct hashfile *f, struct object_entry *entry, { struct packed_git *p = IN_PACK(entry); struct pack_window *w_curs = NULL; - struct revindex_entry *revidx; + uint32_t pos; off_t offset; enum object_type type = oe_type(entry); off_t datalen; @@ -436,10 +436,15 @@ static off_t write_reuse_object(struct hashfile *f, struct object_entry *entry, type, entry_size); offset = entry->in_pack_offset; - revidx = find_pack_revindex(p, offset); - datalen = revidx[1].offset - offset; + if (offset_to_pack_pos(p, offset, &pos) < 0) + die(_("write_reuse_object: could not locate %s, expected at " + "offset %"PRIuMAX" in pack %s"), + oid_to_hex(&entry->idx.oid), (uintmax_t)offset, + p->pack_name); + datalen = pack_pos_to_offset(p, pos + 1) - offset; if (!pack_to_stdout && p->index_version > 1 && - check_pack_crc(p, &w_curs, offset, datalen, revidx->nr)) { + check_pack_crc(p, &w_curs, offset, datalen, + pack_pos_to_index(p, pos))) { error(_("bad packed object CRC for %s"), oid_to_hex(&entry->idx.oid)); unuse_pack(&w_curs); From patchwork Wed Jan 13 22:23:39 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Taylor Blau X-Patchwork-Id: 12018269 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 14DAAC433DB for ; Thu, 14 Jan 2021 02:05:58 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id BEEAF23447 for ; Thu, 14 Jan 2021 02:05:57 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729326AbhAMW1B (ORCPT ); Wed, 13 Jan 2021 17:27:01 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:55060 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1729304AbhAMWYv (ORCPT ); Wed, 13 Jan 2021 17:24:51 -0500 Received: from mail-qk1-x72d.google.com (mail-qk1-x72d.google.com [IPv6:2607:f8b0:4864:20::72d]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 2F3F8C06179F for ; Wed, 13 Jan 2021 14:23:43 -0800 (PST) Received: by mail-qk1-x72d.google.com with SMTP id 19so4430478qkm.8 for ; Wed, 13 Jan 2021 14:23:43 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ttaylorr-com.20150623.gappssmtp.com; s=20150623; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to; bh=th3JkT+WF9nKA4iSS3MEW4HqFzIjPMuj6pDG3bDURq8=; b=slvD1/lsh6fK16re9sTuA+WPCz7A2NDhNSdVeMGRtp7G8jrYX0DffEb7I2t85k/o4j awKlEjqN7rlzQCYNk7IIcEV2UZptuT561OZNQUebIE/THH8+8wSMF8uGXwTskDgq20DV +ExKTSHp5lpr1pWp0SJ+opYgnjx+vPwXqFNZDCzxItqbfNIMB5tkG4fZp6ypH0jwRJFZ Sf47+smJ4E+JkHTnQY7EftKN7jXcHFF+2cq+WxPctPI/BP2Ie94rmQFadrdsLtSoVf3e IFFuJhw9bAODG1zbUH56O+sMqbwNpT0YlLp6+rBAkFQbHTBwv6iL3LqIHCTx83nYLVQt SzyA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to; bh=th3JkT+WF9nKA4iSS3MEW4HqFzIjPMuj6pDG3bDURq8=; b=Sxi/tB2pqAkzPwbW10vhOBmTYzrUqSJN2WVigmMT+TrBLB6cVQoY05h3k/14r5YRyr av+M5AlYnb/a8RlRAQW/HgCx7hScpYEWFs338r4ImAo+WN3LUHP4b/PwnQjIGWrN8E5w JXlokWM69Q/tCbZOKXxmLpc7HsU2u0knReiK9FL4nnZg1QRfDBFLaf0/W3zy14lAHzus gQgh2bDq5hARf+Ag5PU4nREenGK8/JJpmrRqFFzruLVNBhYBweWK4uEk0nXejXHis4do cKkU87mFV/5Xo+IU6OrO6mk5/IRKSbFqLf42cQNUjDc+4xJRYAcs5axjnk0Tb4hL9jui zpSw== X-Gm-Message-State: AOAM532d2WyJNgFc9OFFpH8hv5I2BAhoSnTo7ryvEVIWztwQ28JBvU6d TQjZTp141DpCEdbBGyD6B6VhjRYnzAJErQ== X-Google-Smtp-Source: ABdhPJzIG5mkAPMPNTYW4KTI/i26W5YX8pladSc874mTIFTgNubrSjcHm1Qz2RbshhbQWGfgnbGbJw== X-Received: by 2002:a37:aa57:: with SMTP id t84mr4571145qke.348.1610576622177; Wed, 13 Jan 2021 14:23:42 -0800 (PST) Received: from localhost ([2605:9480:22e:ff10:b172:2e4c:efe4:db53]) by smtp.gmail.com with ESMTPSA id 128sm1933146qki.26.2021.01.13.14.23.41 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 13 Jan 2021 14:23:41 -0800 (PST) Date: Wed, 13 Jan 2021 17:23:39 -0500 From: Taylor Blau To: git@vger.kernel.org Cc: dstolee@microsoft.com, gitster@pobox.com, jrnieder@gmail.com, peff@peff.net Subject: [PATCH v2 03/20] write_reused_pack_one(): convert to new revindex API Message-ID: <7676822a541bfef1861be01dc55d86d3d0cad494.1610576604.git.me@ttaylorr.com> References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Replace direct revindex accesses with calls to 'pack_pos_to_offset()' and 'pack_pos_to_index()'. Signed-off-by: Taylor Blau --- builtin/pack-objects.c | 14 ++++++++++---- 1 file changed, 10 insertions(+), 4 deletions(-) diff --git a/builtin/pack-objects.c b/builtin/pack-objects.c index ab1fd853f1..8e40b19ee8 100644 --- a/builtin/pack-objects.c +++ b/builtin/pack-objects.c @@ -868,8 +868,8 @@ static void write_reused_pack_one(size_t pos, struct hashfile *out, enum object_type type; unsigned long size; - offset = reuse_packfile->revindex[pos].offset; - next = reuse_packfile->revindex[pos + 1].offset; + offset = pack_pos_to_offset(reuse_packfile, pos); + next = pack_pos_to_offset(reuse_packfile, pos + 1); record_reused_object(offset, offset - hashfile_total(out)); @@ -889,11 +889,17 @@ static void write_reused_pack_one(size_t pos, struct hashfile *out, /* Convert to REF_DELTA if we must... */ if (!allow_ofs_delta) { - int base_pos = find_revindex_position(reuse_packfile, base_offset); + uint32_t base_pos; struct object_id base_oid; + if (offset_to_pack_pos(reuse_packfile, base_offset, &base_pos) < 0) + die(_("expected object at offset %"PRIuMAX" " + "in pack %s"), + (uintmax_t)base_offset, + reuse_packfile->pack_name); + nth_packed_object_id(&base_oid, reuse_packfile, - reuse_packfile->revindex[base_pos].nr); + pack_pos_to_index(reuse_packfile, base_pos)); len = encode_in_pack_object_header(header, sizeof(header), OBJ_REF_DELTA, size); From patchwork Wed Jan 13 22:23:43 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Taylor Blau X-Patchwork-Id: 12018325 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 87FCBC4361A for ; Thu, 14 Jan 2021 02:09:55 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 6734F235FA for ; Thu, 14 Jan 2021 02:09:55 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1730124AbhANCJx (ORCPT ); Wed, 13 Jan 2021 21:09:53 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:55130 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726864AbhAMWYv (ORCPT ); Wed, 13 Jan 2021 17:24:51 -0500 Received: from mail-qt1-x82a.google.com (mail-qt1-x82a.google.com [IPv6:2607:f8b0:4864:20::82a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 63147C0617A2 for ; Wed, 13 Jan 2021 14:23:47 -0800 (PST) Received: by mail-qt1-x82a.google.com with SMTP id 2so2287571qtt.10 for ; Wed, 13 Jan 2021 14:23:47 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ttaylorr-com.20150623.gappssmtp.com; s=20150623; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to; bh=bm5o9vq3zI1B3K20xzsSXf0x0nbi7y6y2CnAt9jUucg=; b=a9Bm5s679fMBO50VlMdbJPAD6Eh/8HPkDHXHgV8jqUiXuDKvh6MMNw8GjFiFIVejCT Y9LhlvzYPRQxH2C0NzCVNIAGSGTf7KQwF6km29uy0O2ijpFoPeAbU3nbZ4ZhNfZXbJYF IipaPqQOpbFbiRQYKcUiGXAZi+PhaoaegPlbEtpkIS0ghxeSJ704ZX+AGQLOcNEG1ggb XwSpLsTOhXnaFCOwQkRcm5n3WALM/wMmfnp9b2uWiFCIxMHDE9+liuXq0F9LIIUD5kYc yf4BAracOWh6gBNE5qNYbb+4qTN3IoADmKVCDe4jkhCp1zuzHF0gyoNBkSVj/rtJywPS elBA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to; bh=bm5o9vq3zI1B3K20xzsSXf0x0nbi7y6y2CnAt9jUucg=; b=Rv2dnvXVizgQ3Jx86latKdpSEKgssSnPM8nrVdpzMVqX0mI90uXNlFggB8vO6JMvFs EmMWwFRzazP2UmmVrs47DEDbO8pGgm3Dv61SXcLlGWru+un6MDK8Vlow4ViA+Wt8Wdo9 dgD+/+Z9ahE02EWfxThv2/NDso+QpLWqpFd6w2rvCybii4s+mG/QyvzK+NLQ4QeSsWFw GazW12y1/LQ+UmZwBsLGE01HvjzYin4fh0cu8uVEaX5HbgwAtmRIhbBhWPd9Sb0KwchZ o2RVHDJWHaiSlg5PueGKM5oXpbHppCTmYdhPQP+rYQ/IoJ1a1GZF7e1Ccz7BCMgoMhAO /TSg== X-Gm-Message-State: AOAM531kBxW7Fb8FYfj0UxJqzjoeDcnXfgNLDSKaxHFi6QEf8/NrTKTg k4gkBVTth8RK6YMV0NYeaN5wQyj4Evf9yg== X-Google-Smtp-Source: ABdhPJzcAC8LFQKU0nZtImt3bZ8px4YzMKoiVG0INVpzpG9ZkjCgqIGBc2xWi9q6BXgUFPIiahIQ/Q== X-Received: by 2002:ac8:5a43:: with SMTP id o3mr4452233qta.222.1610576626358; Wed, 13 Jan 2021 14:23:46 -0800 (PST) Received: from localhost ([2605:9480:22e:ff10:b172:2e4c:efe4:db53]) by smtp.gmail.com with ESMTPSA id q73sm1939483qke.16.2021.01.13.14.23.45 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 13 Jan 2021 14:23:45 -0800 (PST) Date: Wed, 13 Jan 2021 17:23:43 -0500 From: Taylor Blau To: git@vger.kernel.org Cc: dstolee@microsoft.com, gitster@pobox.com, jrnieder@gmail.com, peff@peff.net Subject: [PATCH v2 04/20] write_reused_pack_verbatim(): convert to new revindex API Message-ID: References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Replace a direct access to the revindex array with 'pack_pos_to_offset()'. Signed-off-by: Taylor Blau --- builtin/pack-objects.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/builtin/pack-objects.c b/builtin/pack-objects.c index 8e40b19ee8..77ce5583a2 100644 --- a/builtin/pack-objects.c +++ b/builtin/pack-objects.c @@ -952,7 +952,7 @@ static size_t write_reused_pack_verbatim(struct hashfile *out, off_t to_write; written = (pos * BITS_IN_EWORD); - to_write = reuse_packfile->revindex[written].offset + to_write = pack_pos_to_offset(reuse_packfile, written) - sizeof(struct pack_header); /* We're recording one chunk, not one object. */ From patchwork Wed Jan 13 22:23:47 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Taylor Blau X-Patchwork-Id: 12018335 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5FC6DC433DB for ; Thu, 14 Jan 2021 02:12:20 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 18DFF235FA for ; Thu, 14 Jan 2021 02:12:20 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729937AbhANCJ4 (ORCPT ); Wed, 13 Jan 2021 21:09:56 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:55132 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728973AbhAMWYv (ORCPT ); Wed, 13 Jan 2021 17:24:51 -0500 Received: from mail-qt1-x829.google.com (mail-qt1-x829.google.com [IPv6:2607:f8b0:4864:20::829]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 833F0C0617A3 for ; Wed, 13 Jan 2021 14:23:51 -0800 (PST) Received: by mail-qt1-x829.google.com with SMTP id v3so20460qtw.4 for ; Wed, 13 Jan 2021 14:23:51 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ttaylorr-com.20150623.gappssmtp.com; s=20150623; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to; bh=e4IZRUNxseGW/0LfUuiowaMAWmWkA9L1rs66EqMcUyo=; b=QzeRpO2ERaUpBoF5imetwhZerLFbXmNVyOjhO7qVhbLEYE1lQpk5vSMBN4t+e4AYYO SA3GKGwtUtA2jqSYLQDyCQ5uxGgXcja6qYQn+jEJh5NqLTsKIsw6d/TEH/lQJKi0mfOw RQOEUzzcUWzfkPsy+av6amFb51skaL9Q+wJAJBUzV+9Zn/JlaKMAdqHILnoggG8W9usu Hqifx1L4xEsn9kFqwx22u4RlINVzOwYKtLaBW4C5EA2UeTxx85yB+PN2e/ifChUBMlAN dZyqncdTupEq6e73KswTiBPdX1sqmFOZFwpLPi6kvkKu/XLFlOk04gmf2aw+l8hvufBs lqPQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to; bh=e4IZRUNxseGW/0LfUuiowaMAWmWkA9L1rs66EqMcUyo=; b=AhmcMwFQRhfWo1sNwdHVHmYoxGXa28RC6XwBQ2hV2Qp33g1O6RlUbQJx3kCfg3HgDd FhIwpNzAjvdGd7/rv+KO0Bvx1L3Tw3/DSfPiMp36HwEYbUEE3Eq6W9ZcQrtYg9fAXms0 u3qglf/1pZ07aoRQtHpr50gMphP9dYurQJlYWWx703g6997nIxtvBmvSEb5nS7oV+V+B h9sSsrEPC4uPepAPgIs4haFAecAQLN2cPPYphyXMhGdTv8Y5gkkMGiFFGPB9syqaEI85 +mz2euUrxzNfayV8Fb0cQX62NaMP2BwfB+E0+f2VvQDMyul+AhPZ9JRfY0t0BNMOLRRl 46sA== X-Gm-Message-State: AOAM532VHerIa8G9WLECYRe+Q392QXuaKV/jnWr2U+IHjpHC3DUgkTjt Hzber2s3KiNpDk3YIMu7xks1xK0tOKFnzw== X-Google-Smtp-Source: ABdhPJzo/k38DszAJT79BnuripEfmotfJOd9ShhrtCKyPxBPVCdrMWKcrKOrRrTlOjssFLFDWCziPg== X-Received: by 2002:aed:2123:: with SMTP id 32mr4509632qtc.325.1610576630498; Wed, 13 Jan 2021 14:23:50 -0800 (PST) Received: from localhost ([2605:9480:22e:ff10:b172:2e4c:efe4:db53]) by smtp.gmail.com with ESMTPSA id i3sm1815248qtd.95.2021.01.13.14.23.49 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 13 Jan 2021 14:23:49 -0800 (PST) Date: Wed, 13 Jan 2021 17:23:47 -0500 From: Taylor Blau To: git@vger.kernel.org Cc: dstolee@microsoft.com, gitster@pobox.com, jrnieder@gmail.com, peff@peff.net Subject: [PATCH v2 05/20] check_object(): convert to new revindex API Message-ID: <8e93ca38865a9fb9ff9bb264c6db9b6dc14e3029.1610576604.git.me@ttaylorr.com> References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Replace direct accesses to the revindex with calls to 'offset_to_pack_pos()' and 'pack_pos_to_index()'. Since this caller already had some error checking (it can jump to the 'give_up' label if it encounters an error), we can easily check whether or not the provided offset points to an object in the given pack. This error checking existed prior to this patch, too, since the caller checks whether the return value from 'find_pack_revindex()' was NULL or not. Signed-off-by: Taylor Blau --- builtin/pack-objects.c | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/builtin/pack-objects.c b/builtin/pack-objects.c index 77ce5583a2..5b0c4489e2 100644 --- a/builtin/pack-objects.c +++ b/builtin/pack-objects.c @@ -1817,11 +1817,11 @@ static void check_object(struct object_entry *entry, uint32_t object_index) goto give_up; } if (reuse_delta && !entry->preferred_base) { - struct revindex_entry *revidx; - revidx = find_pack_revindex(p, ofs); - if (!revidx) + uint32_t pos; + if (offset_to_pack_pos(p, ofs, &pos) < 0) goto give_up; - if (!nth_packed_object_id(&base_ref, p, revidx->nr)) + if (!nth_packed_object_id(&base_ref, p, + pack_pos_to_index(p, pos))) have_base = 1; } entry->in_pack_header_size = used + used_0; From patchwork Wed Jan 13 22:23:52 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Taylor Blau X-Patchwork-Id: 12018327 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 67787C43603 for ; Thu, 14 Jan 2021 02:09:55 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 4B536235F8 for ; Thu, 14 Jan 2021 02:09:55 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1730085AbhANCJv (ORCPT ); Wed, 13 Jan 2021 21:09:51 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:55130 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1729207AbhAMWZZ (ORCPT ); Wed, 13 Jan 2021 17:25:25 -0500 Received: from mail-qk1-x72b.google.com (mail-qk1-x72b.google.com [IPv6:2607:f8b0:4864:20::72b]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A741EC0617A4 for ; Wed, 13 Jan 2021 14:23:55 -0800 (PST) Received: by mail-qk1-x72b.google.com with SMTP id 186so4451004qkj.3 for ; Wed, 13 Jan 2021 14:23:55 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ttaylorr-com.20150623.gappssmtp.com; s=20150623; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to; bh=eOAfwdgszWE6+IKgpnVgHm09fwsqu7Q3acsEhFfnltQ=; b=QbYuIw+k2+8DM4CaOLUnh5rBCFSJxvyZYTikWKVVchFpr+TrlhwYDnkvWLNwUNuMTE YGXrDevu+wMg5MBCL6UjziknmrcZTjsqtWi4E9Uk6YY56+B9qHM4I6LHM1QAg3Wjq/VH Mevzv7JoFlq508g3qRw5feYRQSBSachGL7HNyVXi2uW8ZgZ/w7qkCh7EH3HvGTCLXa0x 0XJCgvGqqI9CdtQ/7lR1kqi35N8d041lzSEPu7ExafwD1AkM8rM3warFoxdniR4cSgew B1DS0oBx04nUommbKyY3tQz0z8+yqw3tB0H7c0m5p4puiOkGCUYVVhdjacPGMLG6ysQY Ot1g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to; bh=eOAfwdgszWE6+IKgpnVgHm09fwsqu7Q3acsEhFfnltQ=; b=hrvjH+NAct9cMHFi4tnXuFjgdXroGzlmoCqZMSpN4EkowduAXBbLtGUtnhtJ5CKjHE xxkgdZafndHMxD6FnMnyfSibtpOjuOkNvKnWO9kuAgiHEztUzxaRCVM8iR2pVtGmcTa7 gANl89PKuR0fJ5iF9tqGh4s2rJldOnIqq/BP8orqsZ528Lcuql+gG3/RzeXcq92nDZSJ IFzTKWaUt1L29F4w4jTQhfy8wTrwPdt8+aKcFsVMf63PhZv4dsXI6oWiIMyi5QtqSuD8 vHK1Yy/CxGUYvcl4vvySSDGk8EGH/VcdXJhdleAxEzn0Tu+IJTSVugo0NAZpIdgBteKy BokA== X-Gm-Message-State: AOAM5324BQLpoQnfAaG5LeyVRzp2kiihvlzy83HWkOfx6gqmtPCDBfBv 7PzOdCqqdie9OkBZccnHEkelf4pccpyRUQ== X-Google-Smtp-Source: ABdhPJykKNXck0n3GZhUJBjaSLIm3YmLD5qKME6jWGc2U0sMpTuBssXdPjlYymS858ruq5NW5xEIcw== X-Received: by 2002:a37:9b95:: with SMTP id d143mr4232046qke.215.1610576634711; Wed, 13 Jan 2021 14:23:54 -0800 (PST) Received: from localhost ([2605:9480:22e:ff10:b172:2e4c:efe4:db53]) by smtp.gmail.com with ESMTPSA id e19sm1796589qtc.61.2021.01.13.14.23.54 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 13 Jan 2021 14:23:54 -0800 (PST) Date: Wed, 13 Jan 2021 17:23:52 -0500 From: Taylor Blau To: git@vger.kernel.org Cc: dstolee@microsoft.com, gitster@pobox.com, jrnieder@gmail.com, peff@peff.net Subject: [PATCH v2 06/20] bitmap_position_packfile(): convert to new revindex API Message-ID: <084bbf2145735ef1affb0bc051b09dcff8f306ce.1610576604.git.me@ttaylorr.com> References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Replace find_revindex_position() with its counterpart in the new API, offset_to_pack_pos(). Signed-off-by: Taylor Blau --- pack-bitmap.c | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/pack-bitmap.c b/pack-bitmap.c index d88745fb02..d6861ddd4d 100644 --- a/pack-bitmap.c +++ b/pack-bitmap.c @@ -407,11 +407,14 @@ static inline int bitmap_position_extended(struct bitmap_index *bitmap_git, static inline int bitmap_position_packfile(struct bitmap_index *bitmap_git, const struct object_id *oid) { + uint32_t pos; off_t offset = find_pack_entry_one(oid->hash, bitmap_git->pack); if (!offset) return -1; - return find_revindex_position(bitmap_git->pack, offset); + if (offset_to_pack_pos(bitmap_git->pack, offset, &pos) < 0) + return -1; + return pos; } static int bitmap_position(struct bitmap_index *bitmap_git, From patchwork Wed Jan 13 22:23:56 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Taylor Blau X-Patchwork-Id: 12018291 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1C764C433E0 for ; Thu, 14 Jan 2021 02:06:31 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id E06EC23442 for ; Thu, 14 Jan 2021 02:06:30 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729847AbhANCF5 (ORCPT ); Wed, 13 Jan 2021 21:05:57 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:55344 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727396AbhAMWZm (ORCPT ); Wed, 13 Jan 2021 17:25:42 -0500 Received: from mail-qv1-xf2a.google.com (mail-qv1-xf2a.google.com [IPv6:2607:f8b0:4864:20::f2a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id E5099C0617A5 for ; Wed, 13 Jan 2021 14:23:59 -0800 (PST) Received: by mail-qv1-xf2a.google.com with SMTP id l7so1565868qvt.4 for ; Wed, 13 Jan 2021 14:23:59 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ttaylorr-com.20150623.gappssmtp.com; s=20150623; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to; bh=GosB9Yjgb0Kn5JVNBnzEWD9cZazhce4gUyaHBUupnbg=; b=QrvlDMnXOl+m+xX01Mq+NhkjTzQIwfofj3rJYFLsK65rF1inEnlG31riilYNOVjZG+ kO6Xs/vdTFKArGRtpzQU8vjEu9tCKHQ9iA1T2DJf3lcYMxuMr/X38vpWS4+VlF/ly48c PtqIBaKy+P6EGVXfi3UcMPDdXRfCu+iA3+BqTSwP88xIQB8H/m0acx3JFwKh2TIKMpuk QUxTWICMH8DW5zFP4qfPzYLHbcfrp5ed6asv9uY/G+5nX7sM8hRYEmhZy72y/hfAnFnL NKxJcq3HDo6U8s7J6Hc8vrYBYO07Cb827p3DnBaIftZKDgxCrBfy7+m3iQ9znp0/iGFc mAcg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to; bh=GosB9Yjgb0Kn5JVNBnzEWD9cZazhce4gUyaHBUupnbg=; b=FyvT9c6fKXa4rj0rRyDc5tQem4+O3fbssZ7r2eoOK9ZmUNFUrYyFA9MZ0hECGNHWYB kePBe4GjHHETZasy5zbzVGxv45zgKEtPqsnG4E9VHig5kk+oQhuxU5YOcoklRM+26E92 mBQeiSBuh5eMn8L+LrDo0ruR8+f758VjTOU3b3cpYSVfOHdNyYnqhRq02X8mU+zSpLkb beIR8A9uFOfpU1Rpr6jhGcU18whTvcVHb03zUao2NxEo9czWzp8HXqJkzy6EpASMOudo Kiu8Tdnk2WZLjYcsTYOTBEmdKtzRTC+xdjlozKaqOAd4xah8b9M7wf5NiioUkCkupAJi U5uA== X-Gm-Message-State: AOAM532oabY5Jwp4tV54N+RnmbWSOwFdzbyL9zSZ16zkrm2Kvsq+9Y0t e6rsIMhdHC8PEplbxj+qCZxIz0d386eXkw== X-Google-Smtp-Source: ABdhPJyTMomxbJ/gFBs6ZyZIhKAqmetid1wP46P9X2oYYxhKM+a6zaaKAQJybpoMks94VnzR6vshoQ== X-Received: by 2002:a0c:80ca:: with SMTP id 68mr4495603qvb.28.1610576638958; Wed, 13 Jan 2021 14:23:58 -0800 (PST) Received: from localhost ([2605:9480:22e:ff10:b172:2e4c:efe4:db53]) by smtp.gmail.com with ESMTPSA id j30sm1876108qtl.43.2021.01.13.14.23.58 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 13 Jan 2021 14:23:58 -0800 (PST) Date: Wed, 13 Jan 2021 17:23:56 -0500 From: Taylor Blau To: git@vger.kernel.org Cc: dstolee@microsoft.com, gitster@pobox.com, jrnieder@gmail.com, peff@peff.net Subject: [PATCH v2 07/20] show_objects_for_type(): convert to new revindex API Message-ID: <68794e9484d303b832e8a9f7163cf89e2c506479.1610576604.git.me@ttaylorr.com> References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Avoid storing the revindex entry directly, since this structure will soon be removed from the public interface. Instead, store the offset and index position by calling 'pack_pos_to_offset()' and 'pack_pos_to_index()', respectively. Signed-off-by: Taylor Blau --- pack-bitmap.c | 13 +++++++------ 1 file changed, 7 insertions(+), 6 deletions(-) diff --git a/pack-bitmap.c b/pack-bitmap.c index d6861ddd4d..27a7a8ac4c 100644 --- a/pack-bitmap.c +++ b/pack-bitmap.c @@ -711,21 +711,22 @@ static void show_objects_for_type( for (offset = 0; offset < BITS_IN_EWORD; ++offset) { struct object_id oid; - struct revindex_entry *entry; - uint32_t hash = 0; + uint32_t hash = 0, index_pos; + off_t ofs; if ((word >> offset) == 0) break; offset += ewah_bit_ctz64(word >> offset); - entry = &bitmap_git->pack->revindex[pos + offset]; - nth_packed_object_id(&oid, bitmap_git->pack, entry->nr); + index_pos = pack_pos_to_index(bitmap_git->pack, pos + offset); + ofs = pack_pos_to_offset(bitmap_git->pack, pos + offset); + nth_packed_object_id(&oid, bitmap_git->pack, index_pos); if (bitmap_git->hashes) - hash = get_be32(bitmap_git->hashes + entry->nr); + hash = get_be32(bitmap_git->hashes + index_pos); - show_reach(&oid, object_type, 0, hash, bitmap_git->pack, entry->offset); + show_reach(&oid, object_type, 0, hash, bitmap_git->pack, ofs); } } } From patchwork Wed Jan 13 22:24:00 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Taylor Blau X-Patchwork-Id: 12018323 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2CDFAC43219 for ; Thu, 14 Jan 2021 02:09:55 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id F37A5235F8 for ; Thu, 14 Jan 2021 02:09:54 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729819AbhANCJp (ORCPT ); Wed, 13 Jan 2021 21:09:45 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:55132 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1729256AbhAMWZ0 (ORCPT ); Wed, 13 Jan 2021 17:25:26 -0500 Received: from mail-qv1-xf31.google.com (mail-qv1-xf31.google.com [IPv6:2607:f8b0:4864:20::f31]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 5E9E9C061575 for ; Wed, 13 Jan 2021 14:24:04 -0800 (PST) Received: by mail-qv1-xf31.google.com with SMTP id a13so1571743qvv.0 for ; Wed, 13 Jan 2021 14:24:04 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ttaylorr-com.20150623.gappssmtp.com; s=20150623; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to; bh=s/jzWv7ik0siGaWEbLup/aObJEvS0hexzhbxZN1XUSg=; b=qEWVlbxTLbhivMaCYbfkf42S9o/jOO9+xUj87aNBwMIwqTGIcf7vErsiFOui1mlJB8 RqfXD9y9Ygf3OWX0jxIIdssUW4Zt88nyq/7m2ZK/R3nCsVolmKjnyoKoQeQeb439x2bp 7pSFmQGR9G3l5qsET5M8UFGG0oAxp55KaCLBHfNiTsSTT0NFxtP9DuLbmrxp1DAoFuBK 1Vjso2KxTUjc0H00xavcOaUxeIKafdLT5zL46kfdNuwMTUjem65y0+maihtJHh4KrD4V CJvd2XDTQTQXsNIGiIclM/eVezrpFFeNYkQlUS2dDzcFlvB3/JwKzYGbpTnQmxFEOyQH 80CA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to; bh=s/jzWv7ik0siGaWEbLup/aObJEvS0hexzhbxZN1XUSg=; b=OQ5t1wMqrEGHXUrlFCRvu0d9VEpUyRGS8aKZ9smPqSbi2X/pQ7pfDxPGwoQYQDVp9r KwyyNPWwJWtR17XuIev3w+zzXTjjWPaWtTepBoxRQDP3Csa4qHz0R4vU1iDFAuQVSURH 14O8Qo7y+FqM3bN+DOUv7AqlEENJ5Pvb6uZJukI+MfYcJgNwc3XvJRY9UqVAoOw3iLuR mKTyh2o4PMr9PbR0z5CmhYHL7MvnvoyiCeVZnQoWxm4+G5vaD8Sct4XpGxbvSUcM3mXt npOgE7In1nZsG7THCPOPBx+ZkE82ua4NeX+Dlrn/WagimFHfWZKoLyyXe9dto1c2lYxO dfiQ== X-Gm-Message-State: AOAM5305DFKRt6Z0KUf3O4BvfV0Qog5hUGn4syZco8HOZxQdrHfhDTR8 5k+lv5rsEYacicrPzcUd7P5K+1MXomomxw== X-Google-Smtp-Source: ABdhPJzv10b+Pnr7AkzOJ2iPspX/uz3eQ6IHHFprCuEQ5evI+shLIQqandfkuw45dHBy21QtdVsFgA== X-Received: by 2002:a0c:f283:: with SMTP id k3mr4787255qvl.48.1610576643361; Wed, 13 Jan 2021 14:24:03 -0800 (PST) Received: from localhost ([2605:9480:22e:ff10:b172:2e4c:efe4:db53]) by smtp.gmail.com with ESMTPSA id s186sm1887847qka.98.2021.01.13.14.24.02 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 13 Jan 2021 14:24:02 -0800 (PST) Date: Wed, 13 Jan 2021 17:24:00 -0500 From: Taylor Blau To: git@vger.kernel.org Cc: dstolee@microsoft.com, gitster@pobox.com, jrnieder@gmail.com, peff@peff.net Subject: [PATCH v2 08/20] get_size_by_pos(): convert to new revindex API Message-ID: <31ac6f57033f942be5ab4eff96e482a93fda4196.1610576604.git.me@ttaylorr.com> References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Remove another caller that holds onto a 'struct revindex_entry' by replacing the direct indexing with calls to 'pack_pos_to_offset()' and 'pack_pos_to_index()'. Signed-off-by: Taylor Blau --- pack-bitmap.c | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/pack-bitmap.c b/pack-bitmap.c index 27a7a8ac4c..89a528a91b 100644 --- a/pack-bitmap.c +++ b/pack-bitmap.c @@ -835,11 +835,11 @@ static unsigned long get_size_by_pos(struct bitmap_index *bitmap_git, oi.sizep = &size; if (pos < pack->num_objects) { - struct revindex_entry *entry = &pack->revindex[pos]; - if (packed_object_info(the_repository, pack, - entry->offset, &oi) < 0) { + off_t ofs = pack_pos_to_offset(pack, pos); + if (packed_object_info(the_repository, pack, ofs, &oi) < 0) { struct object_id oid; - nth_packed_object_id(&oid, pack, entry->nr); + nth_packed_object_id(&oid, pack, + pack_pos_to_index(pack, pos)); die(_("unable to get size of %s"), oid_to_hex(&oid)); } } else { From patchwork Wed Jan 13 22:24:05 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Taylor Blau X-Patchwork-Id: 12018277 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3CC31C4332E for ; Thu, 14 Jan 2021 02:05:59 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 06AE1235E4 for ; Thu, 14 Jan 2021 02:05:58 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1730111AbhANCFj (ORCPT ); Wed, 13 Jan 2021 21:05:39 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:55378 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727543AbhAMWZw (ORCPT ); Wed, 13 Jan 2021 17:25:52 -0500 Received: from mail-qt1-x82c.google.com (mail-qt1-x82c.google.com [IPv6:2607:f8b0:4864:20::82c]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 076DFC0617A6 for ; Wed, 13 Jan 2021 14:24:10 -0800 (PST) Received: by mail-qt1-x82c.google.com with SMTP id h19so2275469qtq.13 for ; Wed, 13 Jan 2021 14:24:09 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ttaylorr-com.20150623.gappssmtp.com; s=20150623; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to; bh=KYD+b+Qf3QpTk8jEzVklSRxKA6eMCZRnIVnPFQom6eA=; b=bJThc14WGm/MskkkM3eUloWYMTsFG3Ll20UYz9g2Qgvb4Zh6PjMHkA9rEl3SlAXEV5 gjUyMlfSeMesua7zjb612zGwuiImEOhR4k+HFIBeqvHQ0Wj+SAgQTjk/LWRj6IoLoE63 xp33Gtw/0ZQR1GZhJReUqkZXPKjbgQx2s+8xeL1oTLhrH6sVswknvGp3eHet2vPVOLxR YT52W6MJleHVedOXE1mSoCw/dCQ7ojEEBXPDNO/1S+C+8zBpE03oiODF9pAqVRvBapqn qpbQZMWJl3x76fp2EIRGmLhVte6rqhBeIZZ69e7GL5+bHMP447UDb0CsRw70nHK1JcNo q1vw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to; bh=KYD+b+Qf3QpTk8jEzVklSRxKA6eMCZRnIVnPFQom6eA=; b=Rwjbo6ouuFoEHj4+XDI+XA+o5SDoR+fSoCmaNqippwvfq8pptJVtGiqwnO9qJNnQke ohhYjirHpVmAWN4SiayLpKUfWBHnjIAWKm/OIj988auu2tY4GZKQjGRAZii2PCL0OtOC amYu4QzRMGjR1zA8EwhFZml/oush1N1YWfAOUnY5AO0Jd4z4zZDxK4RU+Mh1p4/ZQBoZ IJjFh4ecOtNMcE/Bhi28WbPgIrOFskmY7R5HdFQSzyy+XUb8hWvY0uxs42BRw2GKuFwD 8WiGuTNIw1+xXCt2rj6udFzdzEkHV0kXWNL2spI55qym9TGQTl7ru5t2vj7+3tlwzYhE uF4g== X-Gm-Message-State: AOAM533FoTk6EGSGtc6shVcqvxpqKuh6r+nVcT3mrRxhnDb75Z8WattP 1+PTD+IjLUuLd9oJOGdMdse8bKzS1w5KiA== X-Google-Smtp-Source: ABdhPJwgUWYEOXcckmcfO7/wkgrMPlUUKM9teXFxAsdr3pH0myjgpwLTzxL7iA7mQO5QiKlE4nkDSQ== X-Received: by 2002:aed:2ba5:: with SMTP id e34mr4599597qtd.146.1610576649048; Wed, 13 Jan 2021 14:24:09 -0800 (PST) Received: from localhost ([2605:9480:22e:ff10:b172:2e4c:efe4:db53]) by smtp.gmail.com with ESMTPSA id p75sm1933341qka.72.2021.01.13.14.24.08 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 13 Jan 2021 14:24:08 -0800 (PST) Date: Wed, 13 Jan 2021 17:24:05 -0500 From: Taylor Blau To: git@vger.kernel.org Cc: dstolee@microsoft.com, gitster@pobox.com, jrnieder@gmail.com, peff@peff.net Subject: [PATCH v2 09/20] try_partial_reuse(): convert to new revindex API Message-ID: References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Remove another instance of direct revindex manipulation by calling 'pack_pos_to_offset()' instead (the caller here does not care about the index position of the object at position 'pos'). Note that we cannot just use the existing "offset" variable to store the value we get from pack_pos_to_offset(). It is incremented by unpack_object_header(), but we later need the original value. Since we'll no longer have revindex->offset to read it from, we'll store that in a separate variable ("header" since it points to the entry's header bytes). Signed-off-by: Taylor Blau --- pack-bitmap.c | 13 +++++-------- 1 file changed, 5 insertions(+), 8 deletions(-) diff --git a/pack-bitmap.c b/pack-bitmap.c index 89a528a91b..1fdf7ce20a 100644 --- a/pack-bitmap.c +++ b/pack-bitmap.c @@ -1069,23 +1069,21 @@ static void try_partial_reuse(struct bitmap_index *bitmap_git, struct bitmap *reuse, struct pack_window **w_curs) { - struct revindex_entry *revidx; - off_t offset; + off_t offset, header; enum object_type type; unsigned long size; if (pos >= bitmap_git->pack->num_objects) return; /* not actually in the pack */ - revidx = &bitmap_git->pack->revindex[pos]; - offset = revidx->offset; + offset = header = pack_pos_to_offset(bitmap_git->pack, pos); type = unpack_object_header(bitmap_git->pack, w_curs, &offset, &size); if (type < 0) return; /* broken packfile, punt */ if (type == OBJ_REF_DELTA || type == OBJ_OFS_DELTA) { off_t base_offset; - int base_pos; + uint32_t base_pos; /* * Find the position of the base object so we can look it up @@ -1096,11 +1094,10 @@ static void try_partial_reuse(struct bitmap_index *bitmap_git, * more detail. */ base_offset = get_delta_base(bitmap_git->pack, w_curs, - &offset, type, revidx->offset); + &offset, type, header); if (!base_offset) return; - base_pos = find_revindex_position(bitmap_git->pack, base_offset); - if (base_pos < 0) + if (offset_to_pack_pos(bitmap_git->pack, base_offset, &base_pos) < 0) return; /* From patchwork Wed Jan 13 22:24:27 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Taylor Blau X-Patchwork-Id: 12018289 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 37F80C433DB for ; Thu, 14 Jan 2021 02:06:31 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 08B3C23447 for ; Thu, 14 Jan 2021 02:06:31 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729870AbhANCF7 (ORCPT ); Wed, 13 Jan 2021 21:05:59 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:55376 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727610AbhAMWZv (ORCPT ); Wed, 13 Jan 2021 17:25:51 -0500 Received: from mail-qv1-xf2a.google.com (mail-qv1-xf2a.google.com [IPv6:2607:f8b0:4864:20::f2a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 57DD5C0617A7 for ; Wed, 13 Jan 2021 14:24:31 -0800 (PST) Received: by mail-qv1-xf2a.google.com with SMTP id d11so1553247qvo.11 for ; Wed, 13 Jan 2021 14:24:31 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ttaylorr-com.20150623.gappssmtp.com; s=20150623; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to; bh=YLvmRTXhb3illIomvZeL4AS45p4cH2WTpKemzSExl2M=; b=p7KtcP29e3AzF3cjcTCtRON5fyZHTfPHe617YDyIAz6mAMi6tnzaY3kiq/YnYrrqwo 7IzCkN94IHP3XM/l6E6ZZrIM5a5FLVDY3XVyya+VAlo346qO+QqkQn0gHRkprxDyaNdn 1WtNVEY10J+uRL4In5Ncop1ul/Zbc+HQCNbIsS8Z6pfA1iORpS7nbIDySocqOrH7/6Tv QJtgTX69covomaqHEgDaisOiMTvHERhEar6NxiKS9NU/M9SdW629VqYTjJa+FwbuwEfS xqKmGzj9lD5GjjkuXQQ0l+Gp7XnAMe0A7MvfDP+bKbjdUjBDeOTkURB/TVe97spPDXcx MW3Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to; bh=YLvmRTXhb3illIomvZeL4AS45p4cH2WTpKemzSExl2M=; b=ClmThFIVKiNJL3oq+VxfLs8BLH4y9/hMDOTdmSNFLs30MXQPJSUWZQFCw4+dio1Jtw WMUfWtw2c3bNex2S3zOQIbT90zxO7sM7709cnjEy+Em5uGMMebFllKalqW0Hv1qIgqrh mv66HbYxc8TfvTeoewakHR0TpduDoQETOqyP0wCxiqYZNhdVJTKeraqBUIAn/DSyVAS7 Fd6ZHCa/ONbXibvhjz7/U+qIEgP9IdQoElJR0TtnaiCZ+t7IBF7Z/oSaVqsy4Gh1aWan sC50KnuY/32hH7tDomVJHBAkWyEirpqA/aF7fK0ybgupYuLEVvHheduU1aIYn0wxvyi2 JUzQ== X-Gm-Message-State: AOAM531VVy8rEKiQj7J36AAlAaym/6Ue7SQRTzw3lMiVEP99j5WPAFKT 427p66JnBNp1QMwHOXfkOfe8nnFIjFvYVw== X-Google-Smtp-Source: ABdhPJysfPwPW0htuWs4aZ8s343mIVLGZpiW5G3I/U1ybbFmEYVymHcLgT80Zfb15dxXdAqkulFlJw== X-Received: by 2002:ad4:54a3:: with SMTP id r3mr4640665qvy.26.1610576670441; Wed, 13 Jan 2021 14:24:30 -0800 (PST) Received: from localhost ([2605:9480:22e:ff10:b172:2e4c:efe4:db53]) by smtp.gmail.com with ESMTPSA id k7sm1829834qtg.65.2021.01.13.14.24.29 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 13 Jan 2021 14:24:29 -0800 (PST) Date: Wed, 13 Jan 2021 17:24:27 -0500 From: Taylor Blau To: git@vger.kernel.org Cc: dstolee@microsoft.com, gitster@pobox.com, jrnieder@gmail.com, peff@peff.net Subject: [PATCH v2 10/20] rebuild_existing_bitmaps(): convert to new revindex API Message-ID: <569acdca7f4a5aa25b625280266dfe40dc4fdefe.1610576604.git.me@ttaylorr.com> References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Remove another instance of looking at the revindex directly by instead calling 'pack_pos_to_index()'. Unlike other patches, this caller only cares about the index position of each object in the loop. Signed-off-by: Taylor Blau --- pack-bitmap.c | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-) diff --git a/pack-bitmap.c b/pack-bitmap.c index 1fdf7ce20a..60fe20fb87 100644 --- a/pack-bitmap.c +++ b/pack-bitmap.c @@ -1392,11 +1392,10 @@ uint32_t *create_bitmap_mapping(struct bitmap_index *bitmap_git, for (i = 0; i < num_objects; ++i) { struct object_id oid; - struct revindex_entry *entry; struct object_entry *oe; - entry = &bitmap_git->pack->revindex[i]; - nth_packed_object_id(&oid, bitmap_git->pack, entry->nr); + nth_packed_object_id(&oid, bitmap_git->pack, + pack_pos_to_index(bitmap_git->pack, i)); oe = packlist_find(mapping, &oid); if (oe) From patchwork Wed Jan 13 22:24:32 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Taylor Blau X-Patchwork-Id: 12018293 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 64450C43381 for ; Thu, 14 Jan 2021 02:06:31 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 492BC23442 for ; Thu, 14 Jan 2021 02:06:31 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729883AbhANCGA (ORCPT ); Wed, 13 Jan 2021 21:06:00 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:55380 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1729141AbhAMWZw (ORCPT ); Wed, 13 Jan 2021 17:25:52 -0500 Received: from mail-qk1-x72e.google.com (mail-qk1-x72e.google.com [IPv6:2607:f8b0:4864:20::72e]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 17E1DC0617A9 for ; Wed, 13 Jan 2021 14:24:36 -0800 (PST) Received: by mail-qk1-x72e.google.com with SMTP id 19so4433913qkm.8 for ; Wed, 13 Jan 2021 14:24:36 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ttaylorr-com.20150623.gappssmtp.com; s=20150623; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to; bh=CpNdxqZGfiO4jb4DgI/7fB5wIx6DjqDt1SGCXFIuBIk=; b=toYqjJ4ETox707agmQhYEiNFCoQ2oeLw75uWLX6OeeEeZwcwHSi3SkJlNuRa8HAJkp w4fIRWWls2etuhDWxhQ31v3ry6T4qdBYQAt8b+oJb8VSNQYBNGQ0hQPams65u1VHbAIx +aunNYi7kKCLtg+z1Pw4v8xAdNZCS254+RquC8Z3ckRRJgx373Y6liMvK6lxm7Rpqn1+ SBrjtynV0m43+4KbyLc5RS+ItH4wJ2TeS3hwG6ziO+RCyjWhTPLBfnU41WHPdwWzSZQW 3j2Ouk8WmPZJWcWij2DsHgJrxfWGZ/1xn8Uqx+HfOpamYKmb7qttAGsQrbCGGVgrWvn5 lbwA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to; bh=CpNdxqZGfiO4jb4DgI/7fB5wIx6DjqDt1SGCXFIuBIk=; b=kP04XsUBZ4YIgPe5A+j3e6eMGWXKuRfEO43Qg4t150SctG65wqtsKDiKgKDU5XY3yg W35NsUBtEqg12j96tRtpOQqcTv4F3FiaLbrvR/cRoQrazE0wPnlM9Kc+TLfwerKfRP2+ TZGMnTJJkaWz0UJoKWrtM34wH+Rzh5SNcGVccM1t14fwLc7u4GYVu+lrJiQUOV21rMIk hdosOr//CSvLiuEY/cXtsd+r6+DD8AutkgvWmiDRFbTwZ7vFFx5w/XIcEa7KviVEsCB3 WKpiezdK5lLFYKkc08gYq+xJzizzYZmYHUKWRe+LmqKvCixl2dRI41qjzYW6pcPCSCLk d9cw== X-Gm-Message-State: AOAM5308y/9e8m89mf6Iy+U25W1vR0sLhOsvDNK3rVld8YCKggnW9oHQ HL8/2p6mZKyXu3I8WH1xQBYScLdBWJQOdg== X-Google-Smtp-Source: ABdhPJxh171rad7o8EQwZZx7wW3xPEbWf18/OQpEl0OUL8nZPZxkZuDc+3A13pJQT4aZ2m5aFAS/Qw== X-Received: by 2002:a37:a707:: with SMTP id q7mr4479699qke.284.1610576675010; Wed, 13 Jan 2021 14:24:35 -0800 (PST) Received: from localhost ([2605:9480:22e:ff10:b172:2e4c:efe4:db53]) by smtp.gmail.com with ESMTPSA id m41sm1918766qtc.28.2021.01.13.14.24.34 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 13 Jan 2021 14:24:34 -0800 (PST) Date: Wed, 13 Jan 2021 17:24:32 -0500 From: Taylor Blau To: git@vger.kernel.org Cc: dstolee@microsoft.com, gitster@pobox.com, jrnieder@gmail.com, peff@peff.net Subject: [PATCH v2 11/20] get_delta_base_oid(): convert to new revindex API Message-ID: <98816377248b3112975d49d89a4af7c29d12554e.1610576604.git.me@ttaylorr.com> References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Replace direct accesses to the 'struct revindex' type with a call to 'pack_pos_to_index()'. Likewise drop the old-style 'find_pack_revindex()' with its replacement 'offset_to_pack_pos()' (while continuing to perform the same error checking). Signed-off-by: Taylor Blau --- packfile.c | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/packfile.c b/packfile.c index 86f5c8dbf6..3e3f391949 100644 --- a/packfile.c +++ b/packfile.c @@ -1235,18 +1235,18 @@ static int get_delta_base_oid(struct packed_git *p, oidread(oid, base); return 0; } else if (type == OBJ_OFS_DELTA) { - struct revindex_entry *revidx; + uint32_t base_pos; off_t base_offset = get_delta_base(p, w_curs, &curpos, type, delta_obj_offset); if (!base_offset) return -1; - revidx = find_pack_revindex(p, base_offset); - if (!revidx) + if (offset_to_pack_pos(p, base_offset, &base_pos) < 0) return -1; - return nth_packed_object_id(oid, p, revidx->nr); + return nth_packed_object_id(oid, p, + pack_pos_to_index(p, base_pos)); } else return -1; } From patchwork Wed Jan 13 22:24:36 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Taylor Blau X-Patchwork-Id: 12018295 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8F1B9C433E6 for ; Thu, 14 Jan 2021 02:06:31 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 60C2623447 for ; Thu, 14 Jan 2021 02:06:31 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727422AbhANCGC (ORCPT ); Wed, 13 Jan 2021 21:06:02 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:55382 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1729285AbhAMWZw (ORCPT ); Wed, 13 Jan 2021 17:25:52 -0500 Received: from mail-qk1-x735.google.com (mail-qk1-x735.google.com [IPv6:2607:f8b0:4864:20::735]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 5DA04C0617AA for ; Wed, 13 Jan 2021 14:24:40 -0800 (PST) Received: by mail-qk1-x735.google.com with SMTP id 186so4454105qkj.3 for ; Wed, 13 Jan 2021 14:24:40 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ttaylorr-com.20150623.gappssmtp.com; s=20150623; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to; bh=5SiY02Oy0Xb62PPHOJxVqcbNUHkVa6NVif34C1mP8AM=; b=i/JDNs230ykdhR8B94rAEdFuPz2vpAVfquZakoQDcMO7ok75GGvRt+HUR6fnYeicb/ PSZOSwkQw0Fv35qEk48ZCphwdif9fz9b6QRtKmc2Gq4bJ+3z2TetrAGZ2DaZtCO1sTDL TswlNvkv3hN4AgjtIUOrSQhD7V39ruF1BnJhVBydLx/g3HKkAKrAhRTyClNHOOZ1BRGd R5EgquP344xIQw9WVU0UnQOPHYtNVEUiOzyxy3+c3Un+0v7gBCtVU2xefR4P+o+mf2oS n4eks0fbXEaNwU8d4EKiLr5k3frZ6NFFi34ngUMHTywgwHevQ7opjJQsGFBKijypk6GH dBBA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to; bh=5SiY02Oy0Xb62PPHOJxVqcbNUHkVa6NVif34C1mP8AM=; b=MzxANfaw7GYLkH3+cKYg/mdDQN7HQLrVuAlUPtDjNUR4CY1/iq87aD92xSfJoCzGSC Y6Emk0lnSUYRF0vuZ8rRPKc04N/KmZ1B9h04EFEdFP9gbVe/WJ80yQPQb8ViMn6Mz+vO 95eE3LqEQfz5gbaHDNqBRcZMkDn56cQqA+7y1Clg6KKzJyg8NMHJMbAi5f9buOAGXXm3 hewDj0U3TR35+Ws+/vGWjDnU4gNltIU1LgvdiuhWAPHVYDtNFhk7obHUWR/8XOfKXmDZ KGCiK7P4LhYObpKduW5CgoePy5FE9HpNaZ0nEFSSAEqtBe33mud7nehD7bBPdQk790UV Qwvg== X-Gm-Message-State: AOAM533OhhXu/P1fqQzhznDefz+CypBlojmIOGJsI+OY7mKTokXx26ep DdRZp+9Xt3M22Y04UHPZfuqSFWgC+9CV0A== X-Google-Smtp-Source: ABdhPJwsa/U4Y8MkpRBN4BUmfndWzskqbV4DEJV1iy0166/AqK7NW7JU0uJBp+RVvXs2OlF5o5w4RQ== X-Received: by 2002:a37:9c8a:: with SMTP id f132mr4330669qke.189.1610576679407; Wed, 13 Jan 2021 14:24:39 -0800 (PST) Received: from localhost ([2605:9480:22e:ff10:b172:2e4c:efe4:db53]) by smtp.gmail.com with ESMTPSA id f10sm1887705qtg.27.2021.01.13.14.24.38 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 13 Jan 2021 14:24:38 -0800 (PST) Date: Wed, 13 Jan 2021 17:24:36 -0500 From: Taylor Blau To: git@vger.kernel.org Cc: dstolee@microsoft.com, gitster@pobox.com, jrnieder@gmail.com, peff@peff.net Subject: [PATCH v2 12/20] retry_bad_packed_offset(): convert to new revindex API Message-ID: References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Perform exactly the same conversion as in the previous commit to another caller within 'packfile.c'. Signed-off-by: Taylor Blau --- packfile.c | 7 +++---- 1 file changed, 3 insertions(+), 4 deletions(-) diff --git a/packfile.c b/packfile.c index 3e3f391949..7c37f9ec5c 100644 --- a/packfile.c +++ b/packfile.c @@ -1256,12 +1256,11 @@ static int retry_bad_packed_offset(struct repository *r, off_t obj_offset) { int type; - struct revindex_entry *revidx; + uint32_t pos; struct object_id oid; - revidx = find_pack_revindex(p, obj_offset); - if (!revidx) + if (offset_to_pack_pos(p, obj_offset, &pos) < 0) return OBJ_BAD; - nth_packed_object_id(&oid, p, revidx->nr); + nth_packed_object_id(&oid, p, pack_pos_to_index(p, pos)); mark_bad_packed_object(p, oid.hash); type = oid_object_info(r, &oid, NULL); if (type <= OBJ_NONE) From patchwork Wed Jan 13 22:24:41 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Taylor Blau X-Patchwork-Id: 12018281 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8C876C432C3 for ; Thu, 14 Jan 2021 02:05:59 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 5CD1D235E4 for ; Thu, 14 Jan 2021 02:05:59 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1730108AbhANCFi (ORCPT ); Wed, 13 Jan 2021 21:05:38 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:55384 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1729296AbhAMWZw (ORCPT ); Wed, 13 Jan 2021 17:25:52 -0500 Received: from mail-qv1-xf34.google.com (mail-qv1-xf34.google.com [IPv6:2607:f8b0:4864:20::f34]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 0AD6FC0617AB for ; Wed, 13 Jan 2021 14:24:45 -0800 (PST) Received: by mail-qv1-xf34.google.com with SMTP id h13so1576759qvo.1 for ; Wed, 13 Jan 2021 14:24:45 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ttaylorr-com.20150623.gappssmtp.com; s=20150623; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to; bh=2Sew6UVLwtawBAMq7mXqyFp8UmME/Kil7jsnpb9Kp7k=; b=d5OyHJRm1Ha/2gIhO2wrHZXaEN1u1WPOCieGrKpfV+e3K+usp4azbgdwUZIZutk/y+ Uye1Jh2zgj0WwmpKndofzPXHQkSBeclaLeu7SoqgqJjxUcfhYLZntzqLjA9aeZiOGtga 8JjNF6y1GD8891jeb30rD+oGr9x/LoVHiF6F31lI3wUwFkyNfpxfDMweBb0uAVVF6YDm aXx7TImlOcpjLs6AaXuhmHF6l/fL+Ov3RyiFPMjDcbkKts1LiorSMVJBwrx20V+kYcq/ dFdOtT+tqJaPkDx0XFfRWx4BA588mIEzrqWSekjUx5z7aEfiOrji+bN6LWWPtW8q48P7 iJig== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to; bh=2Sew6UVLwtawBAMq7mXqyFp8UmME/Kil7jsnpb9Kp7k=; b=e1rldWD5MwFqdYj3ZPXRhfTt2QiCKlTGF8J63dX3LsV/SN6oUo13DQ1FgqI9SerXxl Aw6YvJV53166ZICOIxRYEokQB4ESdjEWsU8JA6kQPWT3sg4iKpUe2+uU6g/uA260iAXI Nihc0xUc0Ti3PP0FHhZJ/K8OBBTafgp8yorHp7Fl1GYG+HgqZWxs+OxVX3aYOU/3MM0U NIC8HS7xcdLx9/arfZaA7Pbil7fhJ0fcChc7yJToK3LXmnc/NHUZ2J7vz98vaBb5Mb33 SjM05b0jMEMNwUhICvq8wbP4TNktuvGWcLqzOpc7yqXdFyrFJqKeEFGBfZFsEbnEr6nl rNUA== X-Gm-Message-State: AOAM532rZO5f6gXMf9X/AGWMkLyY19w8X6K/VYxZeLAlkCN8+rZr9P3C ZdXwmgMa6GXYqxqUCPoamcvV3SurCxmRvw== X-Google-Smtp-Source: ABdhPJyJVxg6us9zPC0uRyk/ciMMJ0rx9jdY9TEq0Trp1h5cl7DihWyhum5jV/NaPEKxx0rX7QmfYw== X-Received: by 2002:a0c:df94:: with SMTP id w20mr4399577qvl.33.1610576683997; Wed, 13 Jan 2021 14:24:43 -0800 (PST) Received: from localhost ([2605:9480:22e:ff10:b172:2e4c:efe4:db53]) by smtp.gmail.com with ESMTPSA id l191sm1973333qke.7.2021.01.13.14.24.43 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 13 Jan 2021 14:24:43 -0800 (PST) Date: Wed, 13 Jan 2021 17:24:41 -0500 From: Taylor Blau To: git@vger.kernel.org Cc: dstolee@microsoft.com, gitster@pobox.com, jrnieder@gmail.com, peff@peff.net Subject: [PATCH v2 13/20] packed_object_info(): convert to new revindex API Message-ID: <41b2e00947bdac416e5f599dc50ebf4b0e3e238b.1610576604.git.me@ttaylorr.com> References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Convert another call of 'find_pack_revindex()' to its replacement 'pack_pos_to_offset()'. Likewise: - Avoid manipulating `struct packed_git`'s `revindex` pointer directly by removing the pointer-as-array indexing. - Add an additional guard to check that the offset 'obj_offset()' points to a real object. This should be the case with well-behaved callers to 'packed_object_info()', but isn't guarenteed. Other blocks that fill in various other values from the 'struct object_info' request handle bad inputs by setting the type to 'OBJ_BAD' and jumping to 'out'. Do the same when given a bad offset here. The previous code would have segfaulted when given a bad 'obj_offset' value, since 'find_pack_revindex()' would return 'NULL', and then the line that fills 'oi->disk_sizep' would try to access 'NULL[1]' with a stride of 16 bytes (the width of 'struct revindex_entry)'. Signed-off-by: Taylor Blau --- packfile.c | 11 +++++++++-- 1 file changed, 9 insertions(+), 2 deletions(-) diff --git a/packfile.c b/packfile.c index 7c37f9ec5c..bb4bb14671 100644 --- a/packfile.c +++ b/packfile.c @@ -1537,8 +1537,15 @@ int packed_object_info(struct repository *r, struct packed_git *p, } if (oi->disk_sizep) { - struct revindex_entry *revidx = find_pack_revindex(p, obj_offset); - *oi->disk_sizep = revidx[1].offset - obj_offset; + uint32_t pos; + if (offset_to_pack_pos(p, obj_offset, &pos) < 0) { + error("could not find object at offset %"PRIuMAX" " + "in pack %s", (uintmax_t)obj_offset, p->pack_name); + type = OBJ_BAD; + goto out; + } + + *oi->disk_sizep = pack_pos_to_offset(p, pos + 1) - obj_offset; } if (oi->typep || oi->type_name) { From patchwork Wed Jan 13 22:24:45 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Taylor Blau X-Patchwork-Id: 12018297 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id CA8C5C4332B for ; Thu, 14 Jan 2021 02:06:31 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 828F023442 for ; Thu, 14 Jan 2021 02:06:31 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728951AbhANCGE (ORCPT ); Wed, 13 Jan 2021 21:06:04 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:55438 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1729308AbhAMW0G (ORCPT ); Wed, 13 Jan 2021 17:26:06 -0500 Received: from mail-qt1-x82c.google.com (mail-qt1-x82c.google.com [IPv6:2607:f8b0:4864:20::82c]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 52A2AC0617B0 for ; Wed, 13 Jan 2021 14:24:49 -0800 (PST) Received: by mail-qt1-x82c.google.com with SMTP id c14so2321412qtn.0 for ; Wed, 13 Jan 2021 14:24:49 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ttaylorr-com.20150623.gappssmtp.com; s=20150623; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to; bh=OL4+MFWWUkFQnxJB84R/V4ovdVtSz8QV0aL6M7LI5Uw=; b=nsyVB12vBEmTu9ASaaIexG3gOWdn6iIZFGZKxS+5HL36aBQzstMAVs8uGzkL9klZmP +/7nmOWfo8b90BhNoBuEMAEMGJT4XiUtearGYsNnnF1i2cp+YaJ2rrd6OazCeQ0eEHXs QeXiNLJTeJvIs9cvIpIbcDfwpgGuhGjSi2dsOsf23tQ4MJH8sR+spp7k/TKbGM9wPSb2 j2fUGgCgJB6Mh1YsskXxKWNTs0f9OEKLInxQPPnfAowH3mIGt9mM4/uzJWdBnnsd2ttL KGHBIHR91jhdn7k/g0cXeFnmLIRYACRIRI2OmW4qGGWYPEfGGGufkfm0+vcAB7Io+Yfb betA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to; bh=OL4+MFWWUkFQnxJB84R/V4ovdVtSz8QV0aL6M7LI5Uw=; b=NbmbkevndPmJiSOCwzlJXV0dZ6YLkT+ADmYft7VkYkai5FwwUNgvWb0+pcaSlS7QsI 20LiIFK+4UCZicLv3iE57xC0ITt5TEEbH2YiEr0tvt9cCPlp4p36sRvaEmbAaVsbczGp Ed0z/IKOTn2QZ8ylmduk0oZJWTwMlbnreo2biuh9BoGHcpblX6ZTywpE0khkPvEczr+n 6+Fpc0OaWDN56kPjeWRN3/oCJ0bFIBmqHSRr6wJOX+UeR7QHeVRiT/tIcVHSytNJGsBd sQHwlVlh8TC/2DuhZ/j0fMn/DtpvTvy//W8kp4+emPYDwgmqpT64JwWJToOLJcH90W59 0Obw== X-Gm-Message-State: AOAM532cVXr8RGmICjmaIxYYG9Zo/M0nD9CS9k9vxGfot/RNiSapB/98 hio7M5gD2QLWtXMwU2iN4ApO8Ofq1VaJIg== X-Google-Smtp-Source: ABdhPJyGeLncKpq8CLKbmtPpY55CBrmhbJRuNLZ60HMIxzqVcwdhE8srG/4E+xu1DX+lrV1Tr67vvQ== X-Received: by 2002:ac8:670e:: with SMTP id e14mr4408972qtp.68.1610576688293; Wed, 13 Jan 2021 14:24:48 -0800 (PST) Received: from localhost ([2605:9480:22e:ff10:b172:2e4c:efe4:db53]) by smtp.gmail.com with ESMTPSA id i17sm1875840qtg.77.2021.01.13.14.24.47 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 13 Jan 2021 14:24:47 -0800 (PST) Date: Wed, 13 Jan 2021 17:24:45 -0500 From: Taylor Blau To: git@vger.kernel.org Cc: dstolee@microsoft.com, gitster@pobox.com, jrnieder@gmail.com, peff@peff.net Subject: [PATCH v2 14/20] unpack_entry(): convert to new revindex API Message-ID: <8ad49d231f5e00e258a2e64443cda16626e289c4.1610576604.git.me@ttaylorr.com> References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Remove direct manipulation of the 'struct revindex_entry' type as well as calls to the deprecated API in 'packfile.c:unpack_entry()'. Usual clean-up is performed (replacing '->nr' with calls to 'pack_pos_to_index()' and so on). Add an additional check to make sure that 'obj_offset()' points at a valid object. In the case this check is violated, we cannot call 'mark_bad_packed_object()' because we don't know the OID. At the top of the call stack is do_oid_object_info_extended() (via packed_object_info()), which does mark the object. Signed-off-by: Taylor Blau --- packfile.c | 26 ++++++++++++++++++-------- 1 file changed, 18 insertions(+), 8 deletions(-) diff --git a/packfile.c b/packfile.c index bb4bb14671..936ab3def5 100644 --- a/packfile.c +++ b/packfile.c @@ -1694,11 +1694,21 @@ void *unpack_entry(struct repository *r, struct packed_git *p, off_t obj_offset, } if (do_check_packed_object_crc && p->index_version > 1) { - struct revindex_entry *revidx = find_pack_revindex(p, obj_offset); - off_t len = revidx[1].offset - obj_offset; - if (check_pack_crc(p, &w_curs, obj_offset, len, revidx->nr)) { + uint32_t pack_pos, index_pos; + off_t len; + + if (offset_to_pack_pos(p, obj_offset, &pack_pos) < 0) { + error("could not find object at offset %"PRIuMAX" in pack %s", + (uintmax_t)obj_offset, p->pack_name); + data = NULL; + goto out; + } + + len = pack_pos_to_offset(p, pack_pos + 1) - obj_offset; + index_pos = pack_pos_to_index(p, pack_pos); + if (check_pack_crc(p, &w_curs, obj_offset, len, index_pos)) { struct object_id oid; - nth_packed_object_id(&oid, p, revidx->nr); + nth_packed_object_id(&oid, p, index_pos); error("bad packed object CRC for %s", oid_to_hex(&oid)); mark_bad_packed_object(p, oid.hash); @@ -1781,11 +1791,11 @@ void *unpack_entry(struct repository *r, struct packed_git *p, off_t obj_offset, * This is costly but should happen only in the presence * of a corrupted pack, and is better than failing outright. */ - struct revindex_entry *revidx; + uint32_t pos; struct object_id base_oid; - revidx = find_pack_revindex(p, obj_offset); - if (revidx) { - nth_packed_object_id(&base_oid, p, revidx->nr); + if (!(offset_to_pack_pos(p, obj_offset, &pos))) { + nth_packed_object_id(&base_oid, p, + pack_pos_to_index(p, pos)); error("failed to read delta base object %s" " at offset %"PRIuMAX" from %s", oid_to_hex(&base_oid), (uintmax_t)obj_offset, From patchwork Wed Jan 13 22:24:49 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Taylor Blau X-Patchwork-Id: 12018279 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6B095C43332 for ; Thu, 14 Jan 2021 02:05:59 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 2CCFE235F8 for ; Thu, 14 Jan 2021 02:05:59 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1730113AbhANCFk (ORCPT ); Wed, 13 Jan 2021 21:05:40 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:55444 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1729311AbhAMW0G (ORCPT ); Wed, 13 Jan 2021 17:26:06 -0500 Received: from mail-qk1-x72d.google.com (mail-qk1-x72d.google.com [IPv6:2607:f8b0:4864:20::72d]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 6F7E5C0617BB for ; Wed, 13 Jan 2021 14:24:53 -0800 (PST) Received: by mail-qk1-x72d.google.com with SMTP id w79so4438839qkb.5 for ; Wed, 13 Jan 2021 14:24:53 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ttaylorr-com.20150623.gappssmtp.com; s=20150623; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to; bh=6uTH/FXaX3m7SUUvF9L2p8MarPwkp5TsZ0UURmpTH/Q=; b=AYe6QSgsMCo0XwDAj7nBgB+K1kCJ48IzDiEjLkKfneJ5Dj4mv+yALrP2gEIDzD9Vrb rFgPL6LxbIa2pXT1AKIoXMATYexnG0rly72ecjR1Qvz7/vLplCVeVAe3jUuKvR5SKgBL 5McQC4/KMsgq3GJ3PVQCGbY2/OiatPBEr95+0+iV57GkToBbM+Eon7j0DzkRfoU2OqbM GLdYPR3U0QhZNc7Dny97bean/tmmbNj2quZFS84uL+ulbJNFZdLI4RprGpONCS4dgtRg rIj8hunLTG8cUHas4tboKDcK1yXArnZVBEWyQzvXMyqb7Q3f0Td2j2YM8jv+kfaBy1vg widg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to; bh=6uTH/FXaX3m7SUUvF9L2p8MarPwkp5TsZ0UURmpTH/Q=; b=NR9FJLkOtcP+GYiICA2KFOd1LL4YezD6Nt2sVmawQGimsRaNmx5zuzYv/ywmlAG2KA 4MTmfY3YMRuFeTMYBqRvpP/zTIhg8NGl1TTDBs6a7ArPTqZbJyQvvTJOGsLKf/Gmr/03 7+VYZQg7LNyfd7bpS3NqORFYnWwmkdNNUDj68jM1b1NF3whPa89yKsyGFqPeisO+3Fnl P96cSOAoT4bvGxbohSaUDzLC+452OFdy882hQaxiPL4B2KuuHq7wz7UAgjONEj71sWSg 7nG8jymwWh24nLkrUbN6H4g7OVvGiyx8vW0ifj2gF7/+vH/jsgIBRJ/pq3xCHlgrcrF3 uelQ== X-Gm-Message-State: AOAM533QgN/ejjUjvHkN+X9NH2JGZpYE/KFGKczHIpPorRKwckdjc9Od IPLo4A63V7ina6Omq6cCnAp7Nn37SIsIBg== X-Google-Smtp-Source: ABdhPJyJ1AEV6k10d6kUzH6pCaEybhM+M+onw2+gHhPHjmov6qZDeUknu9ScAOvlMxnVVkzoIhuPNA== X-Received: by 2002:a37:43cc:: with SMTP id q195mr4359989qka.382.1610576692472; Wed, 13 Jan 2021 14:24:52 -0800 (PST) Received: from localhost ([2605:9480:22e:ff10:b172:2e4c:efe4:db53]) by smtp.gmail.com with ESMTPSA id v5sm1967686qkv.64.2021.01.13.14.24.51 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 13 Jan 2021 14:24:51 -0800 (PST) Date: Wed, 13 Jan 2021 17:24:49 -0500 From: Taylor Blau To: git@vger.kernel.org Cc: dstolee@microsoft.com, gitster@pobox.com, jrnieder@gmail.com, peff@peff.net Subject: [PATCH v2 15/20] for_each_object_in_pack(): convert to new revindex API Message-ID: References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Avoid looking at the 'revindex' pointer directly and instead call 'pack_pos_to_index()'. Signed-off-by: Taylor Blau Signed-off-by: Jeff King --- packfile.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/packfile.c b/packfile.c index 936ab3def5..7bb1750934 100644 --- a/packfile.c +++ b/packfile.c @@ -2086,7 +2086,7 @@ int for_each_object_in_pack(struct packed_git *p, struct object_id oid; if (flags & FOR_EACH_OBJECT_PACK_ORDER) - pos = p->revindex[i].nr; + pos = pack_pos_to_index(p, i); else pos = i; From patchwork Wed Jan 13 22:24:54 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Taylor Blau X-Patchwork-Id: 12018283 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id A9F77C43333 for ; Thu, 14 Jan 2021 02:05:59 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 7FFF6235F8 for ; Thu, 14 Jan 2021 02:05:59 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1730101AbhANCFh (ORCPT ); Wed, 13 Jan 2021 21:05:37 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:55376 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1729323AbhAMW1B (ORCPT ); Wed, 13 Jan 2021 17:27:01 -0500 Received: from mail-qv1-xf2f.google.com (mail-qv1-xf2f.google.com [IPv6:2607:f8b0:4864:20::f2f]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A1786C0617BC for ; Wed, 13 Jan 2021 14:24:57 -0800 (PST) Received: by mail-qv1-xf2f.google.com with SMTP id et9so1549611qvb.10 for ; Wed, 13 Jan 2021 14:24:57 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ttaylorr-com.20150623.gappssmtp.com; s=20150623; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to; bh=zCb37iXSFsX9rFmR1LLC+H3EPGfQdLbpmMxbI4AJd3g=; b=acFaaTczZiwXK7pvUFhoCQYnPCw/FUSr3TnJLBMWrlTxp7LDGkL0t8zUZaNx2/cM50 /JiwdfOUmC86MZo8ho1TNi6S68uK3cuTwAMD8PBiNszAqyQd+7DOVdVQjn57AMdqVD5d GlmLqaaDl5Lg6MmkqAijHkbWyAdLOnwNtjEJdpL2q00GvnlmKuDaZ07m07J2G/H4FxrP rvtdMczKOgxVC8S2m9Y2io33dFceA5wtnjMiQ/SmG6/qdB79ccHfpsezLCZ2tk/1sdrW jY5tYXCBrPW6p6mELAschXtlb/bt52CVGIdiFa9ztgNZdhM3FjR4m2Z6btoYaKu6jiQV SDfw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to; bh=zCb37iXSFsX9rFmR1LLC+H3EPGfQdLbpmMxbI4AJd3g=; b=gluNkG8/iXzeeEpdoioZt/94J+r2+Wf3FCnHt+/VkzAhZr+Fml/aG3xp7afKebo4im H7fJOdzdQVQprqROQK4Y+RJiWJvOvf7w72IxB+VwRcQKwFsX5GS3MW+O7hZE0uiaLuNb r+pZ1i/q0GnyACE+NyxGzrHi0bDkiP/p6XBiL1ylPuQilexo8LAT1mPma3BE0DKZ330p iGHtsTUb5oC1/nlEYEDAlCO/zGs84ECC1+AMhKt4T/ir8BTXy9J42Jq8Ts+Je5/cf+tN xiVw5hRz1q55V9ePd9FtSyooZQlRwgDDlFhPNB5Q8x/q8fMlwAKk9bdF9xUtx36iN6Ed t+oA== X-Gm-Message-State: AOAM531b000LuzvSp1YzkmmZqIH1VTA/5zE6dog/T1mos6W5cxJTGEMk z8aa16ZM/sA22Zls/lqkzUpmAK1VZHtK6A== X-Google-Smtp-Source: ABdhPJy9Apfghf30sKN+Vx0wEssRs3KA6qz6UN+iw6S/hl+HJsUxZglSXW/1fn7sNe4qbEwpltNUPA== X-Received: by 2002:a05:6214:370:: with SMTP id t16mr4705154qvu.22.1610576696682; Wed, 13 Jan 2021 14:24:56 -0800 (PST) Received: from localhost ([2605:9480:22e:ff10:b172:2e4c:efe4:db53]) by smtp.gmail.com with ESMTPSA id v5sm1967777qkv.64.2021.01.13.14.24.55 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 13 Jan 2021 14:24:56 -0800 (PST) Date: Wed, 13 Jan 2021 17:24:54 -0500 From: Taylor Blau To: git@vger.kernel.org Cc: dstolee@microsoft.com, gitster@pobox.com, jrnieder@gmail.com, peff@peff.net Subject: [PATCH v2 16/20] builtin/gc.c: guess the size of the revindex Message-ID: References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org 'estimate_repack_memory()' takes into account the amount of memory required to load the reverse index in memory by multiplying the assumed number of objects by the size of the 'revindex_entry' struct. Prepare for hiding the definition of 'struct revindex_entry' by removing a 'sizeof()' of that type from outside of pack-revindex.c. Instead, guess that one off_t and one uint32_t are required per object. Strictly speaking, this is a worse guess than asking for 'sizeof(struct revindex_entry)' directly, since the true size of this struct is 16 bytes with padding on the end of the struct in order to align the offset field. But, this is an approximation anyway, and it does remove a use of the 'struct revindex_entry' from outside of pack-revindex internals. Signed-off-by: Taylor Blau --- builtin/gc.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/builtin/gc.c b/builtin/gc.c index 4c24f41852..c60811f212 100644 --- a/builtin/gc.c +++ b/builtin/gc.c @@ -301,7 +301,7 @@ static uint64_t estimate_repack_memory(struct packed_git *pack) /* and then obj_hash[], underestimated in fact */ heap += sizeof(struct object *) * nr_objects; /* revindex is used also */ - heap += sizeof(struct revindex_entry) * nr_objects; + heap += (sizeof(off_t) + sizeof(uint32_t)) * nr_objects; /* * read_sha1_file() (either at delta calculation phase, or * writing phase) also fills up the delta base cache From patchwork Wed Jan 13 22:24:58 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Taylor Blau X-Patchwork-Id: 12018287 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 27CF2C43603 for ; Thu, 14 Jan 2021 02:06:00 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id E89BE235E4 for ; Thu, 14 Jan 2021 02:05:59 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1730088AbhANCFg (ORCPT ); Wed, 13 Jan 2021 21:05:36 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:55378 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1729327AbhAMW1C (ORCPT ); Wed, 13 Jan 2021 17:27:02 -0500 Received: from mail-qk1-x733.google.com (mail-qk1-x733.google.com [IPv6:2607:f8b0:4864:20::733]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B7E40C0617BD for ; Wed, 13 Jan 2021 14:25:01 -0800 (PST) Received: by mail-qk1-x733.google.com with SMTP id d14so4411837qkc.13 for ; Wed, 13 Jan 2021 14:25:01 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ttaylorr-com.20150623.gappssmtp.com; s=20150623; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to; bh=v/cn6izEdOXRJ7vzBjk5AFjC0JvkuXpqARQkKmAiPQg=; b=Aryq8vRIkB2qiZ2nV85lkPZ2E91nY7JH+EQ7yqN+R7+woY71RrYvv27Q12OJLJo0SB tfe/4BoXMrj2prOlWPAuXMC/M8Afuys8SrWh8EKPfDUAq36gj4oln9NXrdXhG2TT9QAO z/Rtd7TJq3DWlIBFNNL7sRTWZN3duUjD2exQcU5s/vufRY+lYqZZZODQ4P8OCsUwWct2 zEfqAKOWakkHHy47Qvw8/UoMyAy66IWU4okQedB7QfNhhGs93WCoKUIkvST6+HgDj2Hm vYQtsNQA18WcQ1pA3XfYPyAYWPrYsqLPlw7YB9tM2NGOJHCHQ8gkqa95jai2r0PxO902 FgvA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to; bh=v/cn6izEdOXRJ7vzBjk5AFjC0JvkuXpqARQkKmAiPQg=; b=K3QQlxR9A7q+8Ucyc2//blQqObbMLICTEeYEqDIY2TN54Rk6eWoPx08KlmpAjlgU0y e0QI1AHfW41M1ViZagPyvugkh4fUdAKQLdW4tRxI9A/324jzaSxzR8vcSt+tUqHPmYPk JmsZYkTr9hTn4F8rHS3N7uKMPktUd26+NH+A3D8aAqIHpJxbTEOIiFUqdtUlWN0hAKF/ QtEqAtB8Nf+ah7OiP73nf3pmy9dUxzQQfRQ2nkQw3OEJEDYtlaosXyoj4/ccDP/+bHkf yKN4i9UrxW0DmSERWUXzYmXcyHvvcIoz0btYHVyTzJB554beVbJrpKOjEcHIcb0t3WCG //WA== X-Gm-Message-State: AOAM530HxTK/IVUhH0GGu4qhDxjPLny90iokeYVaCI7qiqQJ6qXrxXS1 86hBFdndcr5dK2oRSUmJjcVMQRlTwX1Hmw== X-Google-Smtp-Source: ABdhPJzCighb9c2RAMOH7NaZiyUkVpP8ppgQ0U2RDyf6InUFDNtMJrE9DnTUBb8iaDjw+YLUGihWNA== X-Received: by 2002:a37:7641:: with SMTP id r62mr4456966qkc.227.1610576700795; Wed, 13 Jan 2021 14:25:00 -0800 (PST) Received: from localhost ([2605:9480:22e:ff10:b172:2e4c:efe4:db53]) by smtp.gmail.com with ESMTPSA id q32sm1988934qtb.0.2021.01.13.14.25.00 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 13 Jan 2021 14:25:00 -0800 (PST) Date: Wed, 13 Jan 2021 17:24:58 -0500 From: Taylor Blau To: git@vger.kernel.org Cc: dstolee@microsoft.com, gitster@pobox.com, jrnieder@gmail.com, peff@peff.net Subject: [PATCH v2 17/20] pack-revindex: remove unused 'find_pack_revindex()' Message-ID: <67d14da04a147e1805807e405cbecf91e831f5cc.1610576604.git.me@ttaylorr.com> References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Now that no callers of 'find_pack_revindex()' remain, remove the function's declaration and implementation entirely. Signed-off-by: Taylor Blau --- pack-revindex.c | 15 --------------- pack-revindex.h | 2 -- 2 files changed, 17 deletions(-) diff --git a/pack-revindex.c b/pack-revindex.c index 0ca3b54b45..16baafb281 100644 --- a/pack-revindex.c +++ b/pack-revindex.c @@ -189,21 +189,6 @@ int find_revindex_position(struct packed_git *p, off_t ofs) return -1; } -struct revindex_entry *find_pack_revindex(struct packed_git *p, off_t ofs) -{ - int pos; - - if (load_pack_revindex(p)) - return NULL; - - pos = find_revindex_position(p, ofs); - - if (pos < 0) - return NULL; - - return p->revindex + pos; -} - int offset_to_pack_pos(struct packed_git *p, off_t ofs, uint32_t *pos) { int ret; diff --git a/pack-revindex.h b/pack-revindex.h index 5a218aaa66..f7094ba9a5 100644 --- a/pack-revindex.h +++ b/pack-revindex.h @@ -30,8 +30,6 @@ struct revindex_entry { int load_pack_revindex(struct packed_git *p); int find_revindex_position(struct packed_git *p, off_t ofs); -struct revindex_entry *find_pack_revindex(struct packed_git *p, off_t ofs); - /* * offset_to_pack_pos converts an object offset to a pack position. This * function returns zero on success, and a negative number otherwise. The From patchwork Wed Jan 13 22:25:02 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Taylor Blau X-Patchwork-Id: 12018301 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id CD1E0C433E0 for ; Thu, 14 Jan 2021 02:07:19 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 8A3AF235E4 for ; Thu, 14 Jan 2021 02:07:19 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726565AbhANCFf (ORCPT ); Wed, 13 Jan 2021 21:05:35 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:55344 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1729321AbhAMW1B (ORCPT ); Wed, 13 Jan 2021 17:27:01 -0500 Received: from mail-qk1-x731.google.com (mail-qk1-x731.google.com [IPv6:2607:f8b0:4864:20::731]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 15633C0617A2 for ; Wed, 13 Jan 2021 14:25:06 -0800 (PST) Received: by mail-qk1-x731.google.com with SMTP id w79so4439642qkb.5 for ; Wed, 13 Jan 2021 14:25:06 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ttaylorr-com.20150623.gappssmtp.com; s=20150623; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to; bh=gfK+1MKqel64GNCY5eNaDGEceywQu/Dwr9adToVw6K8=; b=YpS+48OFOEqkPmpsGnMno03EwaaHi1vPNNQ9c+YM0lwvpxjsLgAeyDRyR4f+iE2Sol BUFLQRmS61GfiNISF0Oyez+RQuSu5Jv9gwV9XNqA3y39Th6gJYn9Hflx3sIg0U7aOYTy 5zpheitO45qGQZ6VG2yaCqKvg2mOnIa+jDvOP1/qMZLavygZyeP5xTruaFCrZVTacMMA cKHNuDN+AGZZeSddBBCegVd9kS7yP2hQKVOtOkLTkdisIUej2Igm7QHCM9j3GKArkGgg LfbQ/+n1Zqkikmh+eB0aSbZdeTCsJctVSeo75DP8IUipbTgVEWf8cKK307SISNs7bjH3 lPxQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to; bh=gfK+1MKqel64GNCY5eNaDGEceywQu/Dwr9adToVw6K8=; b=ZHHY2hsAmRH22QR/GIV8UXmTYIXyrIi2X0oqbCcVhFLFclIKh0LknCoR69wfDy+czM sXLJNNqiUhlQDMC66nuyYbP2VhFniIzpqFXzlvh9BvTf+HODVdqX2wrGEjsEoK0brUBH C+NPabIWFlu2icyLwV/EEoAQYQu08pnEs3aX0d2UF69GjiukmegTwxcRt/UdHdGG7Utk OKwcHC1UeEkYPba82XRHRqV/jg6kzpAKNb2JRVw+S47CJrPso9WW7jbTqU4UaauZJX18 oJKpIFIsEtY2jly50LVSaqOWhH5TVk8SD6EPzDESWzEib8jA4jNkpMEBonbPE1Xfv0rh e+hA== X-Gm-Message-State: AOAM533T4/JQhmSiIlnJ/NORLngeQYTlWlB6mA09F+TVjJIj3jFjS+xn SrHBvuyPA7bh9dV5VuHi6CHUNFUUeyGbFg== X-Google-Smtp-Source: ABdhPJwnxrEbK2XitcbFHL8alNFQSdwDkrNTi9IS4kZ3nliBylkQ37CVWnaCCB7yYRSju/2Lxe3YRw== X-Received: by 2002:a37:43cc:: with SMTP id q195mr4360698qka.382.1610576705078; Wed, 13 Jan 2021 14:25:05 -0800 (PST) Received: from localhost ([2605:9480:22e:ff10:b172:2e4c:efe4:db53]) by smtp.gmail.com with ESMTPSA id 141sm1924390qkn.53.2021.01.13.14.25.04 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 13 Jan 2021 14:25:04 -0800 (PST) Date: Wed, 13 Jan 2021 17:25:02 -0500 From: Taylor Blau To: git@vger.kernel.org Cc: dstolee@microsoft.com, gitster@pobox.com, jrnieder@gmail.com, peff@peff.net Subject: [PATCH v2 18/20] pack-revindex: remove unused 'find_revindex_position()' Message-ID: <3b5c92be684b95f04cbe224c791d87657be9ff79.1610576604.git.me@ttaylorr.com> References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Now that all 'find_revindex_position()' callers have been removed (and converted to the more descriptive 'offset_to_pack_pos()'), it is almost safe to get rid of 'find_revindex_position()' entirely. Almost, except for the fact that 'offset_to_pack_pos()' calls 'find_revindex_position()'. Inline 'find_revindex_position()' into 'offset_to_pack_pos()', and then remove 'find_revindex_position()' entirely. This is a straightforward refactoring with one minor snag. 'offset_to_pack_pos()' used to load the index before calling 'find_revindex_position()'. That means that by the time 'find_revindex_position()' starts executing, 'p->num_objects' can be safely read. After inlining, be careful to not read 'p->num_objects' until _after_ 'load_pack_revindex()' (which loads the index as a side-effect) has been called. Another small fix that is included is converting the upper- and lower-bounds to be unsigned's instead of ints. This dates back to 92e5c77c37 (revindex: export new APIs, 2013-10-24)--ironically, the last time we introduced new APIs here--but this unifies the types. Signed-off-by: Taylor Blau --- pack-revindex.c | 31 ++++++++++++------------------- pack-revindex.h | 1 - 2 files changed, 12 insertions(+), 20 deletions(-) diff --git a/pack-revindex.c b/pack-revindex.c index 16baafb281..282fe92640 100644 --- a/pack-revindex.c +++ b/pack-revindex.c @@ -169,16 +169,23 @@ int load_pack_revindex(struct packed_git *p) return 0; } -int find_revindex_position(struct packed_git *p, off_t ofs) +int offset_to_pack_pos(struct packed_git *p, off_t ofs, uint32_t *pos) { - int lo = 0; - int hi = p->num_objects + 1; - const struct revindex_entry *revindex = p->revindex; + unsigned lo, hi; + const struct revindex_entry *revindex; + + if (load_pack_revindex(p) < 0) + return -1; + + lo = 0; + hi = p->num_objects + 1; + revindex = p->revindex; do { const unsigned mi = lo + (hi - lo) / 2; if (revindex[mi].offset == ofs) { - return mi; + *pos = mi; + return 0; } else if (ofs < revindex[mi].offset) hi = mi; else @@ -189,20 +196,6 @@ int find_revindex_position(struct packed_git *p, off_t ofs) return -1; } -int offset_to_pack_pos(struct packed_git *p, off_t ofs, uint32_t *pos) -{ - int ret; - - if (load_pack_revindex(p) < 0) - return -1; - - ret = find_revindex_position(p, ofs); - if (ret < 0) - return ret; - *pos = ret; - return 0; -} - uint32_t pack_pos_to_index(struct packed_git *p, uint32_t pos) { if (!p->revindex) diff --git a/pack-revindex.h b/pack-revindex.h index f7094ba9a5..746776be7f 100644 --- a/pack-revindex.h +++ b/pack-revindex.h @@ -28,7 +28,6 @@ struct revindex_entry { * given pack, returning zero on success and a negative value otherwise. */ int load_pack_revindex(struct packed_git *p); -int find_revindex_position(struct packed_git *p, off_t ofs); /* * offset_to_pack_pos converts an object offset to a pack position. This From patchwork Wed Jan 13 22:25:06 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Taylor Blau X-Patchwork-Id: 12018285 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 16046C43217 for ; Thu, 14 Jan 2021 02:06:00 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id C38EF2360D for ; Thu, 14 Jan 2021 02:05:59 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1730097AbhANCFh (ORCPT ); Wed, 13 Jan 2021 21:05:37 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:55384 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1729324AbhAMW1B (ORCPT ); Wed, 13 Jan 2021 17:27:01 -0500 Received: from mail-qv1-xf2e.google.com (mail-qv1-xf2e.google.com [IPv6:2607:f8b0:4864:20::f2e]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 2D1BCC0617BE for ; Wed, 13 Jan 2021 14:25:10 -0800 (PST) Received: by mail-qv1-xf2e.google.com with SMTP id h1so1543504qvy.12 for ; Wed, 13 Jan 2021 14:25:10 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ttaylorr-com.20150623.gappssmtp.com; s=20150623; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to; bh=1BnFTlplhhpGCXeXRc1kNQSrwdBZw4WL4WJUUVbtRd8=; b=sMZvlp94ilZZcH/niq4bnV0Nlq4v2lm+lAcCqkEz4MtcJ6scEvBIkB/9AzjzS+IfF5 LrGOQyf9mmBN7/OU/hXtdk9QRy32BBGcp/ZXy1cpj4GG5KFrondK955N6GZtk5jlXOZa /XnNP1bx6qS1XdxExvFfQkBDf7wxzKnoyhv/K/1QXaXX8a7ewtjklgrmL4VdTscBWfDd jOVcPXwiv0RuvVVT7KpNBPhNXwOpI5uE1hCbFxBaMLaHYE9I4+RvBOTh3r60R+/DIjDT yetHBhr/iAkp/D6ukD8G3qtwpsF2Eu1wc3R21Pck2Es0Gql9hqyAoWCJQ4leiy/FPEDE Y0KQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to; bh=1BnFTlplhhpGCXeXRc1kNQSrwdBZw4WL4WJUUVbtRd8=; b=Ive1D5K+UuXsKVQgBW+E9rkot8ey95+Kv4wqKscus6G7kASEl7PO9UGpmrv4ClMpQs 3hZpSHMXBXUzUeeJpbWOmNQymcUAYL609OaviG9VP6p1IQ9K7GuSqKLqC4ZbqS+vpuL8 wCQ0TRg9w2OmyfNhtHUtTmm6CqXcBHNkcBAAW8sAIF+oRRX40qzFu3G6ZYwHu94Ctv22 +jPOhtxqa1NuR0Crt2BWchPG6lRvKhV2NuE+EfPYYKu6KZPhvpKXwK33gNGk7BSLwT/e 6E/dosizV6pC0cu+GOMjfnhsD4XTpRKlcH8bnwA9NoGcIzowtmtLtVxe1OsInVPrOapL k4Ww== X-Gm-Message-State: AOAM533jJV4/wFxIrKgO9/DaXdx5LW77cnf3l0MNZ9JnFlHR31tikIlI ZcJxu6kuvPpRQzHLI97gVfaYy/umbPfTrw== X-Google-Smtp-Source: ABdhPJwRdy7dHTypjI9oqBL3Z/ck/MqDATE6RKnlwYmhwr1hqZCNfPqHblIqQ3eE4NlbnkRlGRUNfA== X-Received: by 2002:a05:6214:1110:: with SMTP id e16mr4591945qvs.62.1610576709162; Wed, 13 Jan 2021 14:25:09 -0800 (PST) Received: from localhost ([2605:9480:22e:ff10:b172:2e4c:efe4:db53]) by smtp.gmail.com with ESMTPSA id k7sm1830808qtg.65.2021.01.13.14.25.08 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 13 Jan 2021 14:25:08 -0800 (PST) Date: Wed, 13 Jan 2021 17:25:06 -0500 From: Taylor Blau To: git@vger.kernel.org Cc: dstolee@microsoft.com, gitster@pobox.com, jrnieder@gmail.com, peff@peff.net Subject: [PATCH v2 19/20] pack-revindex: hide the definition of 'revindex_entry' Message-ID: References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Now that all spots outside of pack-revindex.c that reference 'struct revindex_entry' directly have been removed, it is safe to hide the implementation by moving it from pack-revindex.h to pack-revindex.c. Signed-off-by: Taylor Blau --- pack-revindex.c | 5 +++++ pack-revindex.h | 5 ----- 2 files changed, 5 insertions(+), 5 deletions(-) diff --git a/pack-revindex.c b/pack-revindex.c index 282fe92640..a508d5f0a4 100644 --- a/pack-revindex.c +++ b/pack-revindex.c @@ -3,6 +3,11 @@ #include "object-store.h" #include "packfile.h" +struct revindex_entry { + off_t offset; + unsigned int nr; +}; + /* * Pack index for existing packs give us easy access to the offsets into * corresponding pack file where each object's data starts, but the entries diff --git a/pack-revindex.h b/pack-revindex.h index 746776be7f..6e0320b08b 100644 --- a/pack-revindex.h +++ b/pack-revindex.h @@ -18,11 +18,6 @@ struct packed_git; -struct revindex_entry { - off_t offset; - unsigned int nr; -}; - /* * load_pack_revindex populates the revindex's internal data-structures for the * given pack, returning zero on success and a negative value otherwise. From patchwork Wed Jan 13 22:25:10 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Taylor Blau X-Patchwork-Id: 12018275 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 17342C43331 for ; Thu, 14 Jan 2021 02:05:59 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id D6B8B235F8 for ; Thu, 14 Jan 2021 02:05:58 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1730104AbhANCFh (ORCPT ); Wed, 13 Jan 2021 21:05:37 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:55132 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1729320AbhAMW1B (ORCPT ); Wed, 13 Jan 2021 17:27:01 -0500 Received: from mail-qk1-x731.google.com (mail-qk1-x731.google.com [IPv6:2607:f8b0:4864:20::731]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 2C3C4C061795 for ; Wed, 13 Jan 2021 14:25:14 -0800 (PST) Received: by mail-qk1-x731.google.com with SMTP id 143so4408693qke.10 for ; Wed, 13 Jan 2021 14:25:14 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ttaylorr-com.20150623.gappssmtp.com; s=20150623; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to; bh=ygOOxZvwT4fpebBNERIgQabv33pbsiBcH0bco8d1HdE=; b=ZhJE/VlsScZDMtQ1zZ+V38BYhnG2aXL9tykAAM627F6cGxMEnrxRejw0bELlti8B+R /2JI9nWHzmMMTRBKZVaa/BtvJYSoedKpdwNW5xR7fyiUpuwznbDzl5A7ywGl1NMz6l9u EC2fgXhUH9IoX7qku7h2WOC6WWyDQInonkSjej/liQOSb/cXmuK+UNkvmLBiZH1GkbCt rmRcW2VpDboAjLmX3Kv0uqDafAgMRzhaYS+Ubmg1pZ5by1VXVQvMnTfFBIjKz6kzSmaH Wvrmd2RHGwJ6bYl5pNSaF+cGKQ2aYaGBTdoe5RaXvPzREtlQk2HbpUNTDmWMv6dbipYt KbTA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to; bh=ygOOxZvwT4fpebBNERIgQabv33pbsiBcH0bco8d1HdE=; b=eU0dECqcFm3Pw5VTgGpk4/8+OyKkip5FvN9SNW/Ce2PAby+CVRfcFYamXcsCi8kjjI GDmeTr8ic/2FInlrknwnut9stNOUYg42L7oSfdykx9p56tdivz2Hx89HqYC4mj1lyYBK V4kdez3metjTzybaLt7wUkGDi22wpe0j/JnpwTvzSTZg0M5ftwmlfYLvv+1rN0XujDRW tljKI2rE18+YDBEED4/KWrOHOOq1L/UnPQVXSt1OkdBxVtw5GIKiLVpRWx4p+hKwOV/V W0FmIK76xRji2Ape3Cj+9ncT+L2+cLGFZPp+U/w+6R+HWLWlbLpRqMjjDpHQBqCKRzam pV5g== X-Gm-Message-State: AOAM531Zw4gXPQeUAAflMZMdOVGe+WadNFnqGJcNYynPyMR/MQZXJc/F t+LlBVwwBobevdLTmZ1PGwYSCWGxYGJhjw== X-Google-Smtp-Source: ABdhPJyjHtGtzmonZEVi0zIIYmI/fd7tqg8E1R3EtoiMJilMfuX9KLEeL16/aPGNmYui68Vn6pFYsA== X-Received: by 2002:a37:b083:: with SMTP id z125mr4329061qke.246.1610576713182; Wed, 13 Jan 2021 14:25:13 -0800 (PST) Received: from localhost ([2605:9480:22e:ff10:b172:2e4c:efe4:db53]) by smtp.gmail.com with ESMTPSA id o201sm1984703qke.60.2021.01.13.14.25.12 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 13 Jan 2021 14:25:12 -0800 (PST) Date: Wed, 13 Jan 2021 17:25:10 -0500 From: Taylor Blau To: git@vger.kernel.org Cc: dstolee@microsoft.com, gitster@pobox.com, jrnieder@gmail.com, peff@peff.net Subject: [PATCH v2 20/20] pack-revindex.c: avoid direct revindex access in 'offset_to_pack_pos()' Message-ID: <8400ff6c9615b4c999b198c46b2e673ec0f2b14f.1610576604.git.me@ttaylorr.com> References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org To prepare for on-disk reverse indexes, remove a spot in 'offset_to_pack_pos()' that looks at the 'revindex' array in 'struct packed_git'. Even though this use of the revindex pointer is within pack-revindex.c, this clean up is still worth doing. Since the 'revindex' pointer will be NULL when reading from an on-disk reverse index (instead the 'revindex_data' pointer will be mmaped to the 'pack-*.rev' file), this call-site would have to include a conditional to lookup the offset for position 'mi' each iteration through the search. So instead of open-coding 'pack_pos_to_offset()', call it directly from within 'offset_to_pack_pos()'. Signed-off-by: Taylor Blau --- pack-revindex.c | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/pack-revindex.c b/pack-revindex.c index a508d5f0a4..5e69bc7372 100644 --- a/pack-revindex.c +++ b/pack-revindex.c @@ -177,21 +177,21 @@ int load_pack_revindex(struct packed_git *p) int offset_to_pack_pos(struct packed_git *p, off_t ofs, uint32_t *pos) { unsigned lo, hi; - const struct revindex_entry *revindex; if (load_pack_revindex(p) < 0) return -1; lo = 0; hi = p->num_objects + 1; - revindex = p->revindex; do { const unsigned mi = lo + (hi - lo) / 2; - if (revindex[mi].offset == ofs) { + off_t got = pack_pos_to_offset(p, mi); + + if (got == ofs) { *pos = mi; return 0; - } else if (ofs < revindex[mi].offset) + } else if (ofs < got) hi = mi; else lo = mi + 1;