From patchwork Tue Jun 11 18:24:33 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Victoria Dye X-Patchwork-Id: 13694103 Received: from mail-wr1-f43.google.com (mail-wr1-f43.google.com [209.85.221.43]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id BC0CD2628D for ; Tue, 11 Jun 2024 18:24:52 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.221.43 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1718130294; cv=none; b=V90UAr9jis1Wcr5CHe1toPEz5SN8cPduv5xKPJ0CKtasI7B9FlKdfXHw6li7L2w8wXB+8stryDv/464qHmyOsSgNUvPRaoiirplmdKpiRODNS54GprruqXZff/EuR/qFbrdOmjpClYlOM2JoRA8fOuyduAG2cwUORhavHIuh/Q4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1718130294; c=relaxed/simple; bh=fBat4Yp3gYivyKXamk3+/10ToBE4v9Q8CQR4fkIkCNw=; h=Message-Id:In-Reply-To:References:From:Date:Subject:Content-Type: MIME-Version:To:Cc; b=sbxuJxO++NAlWDYtcFEFZMbBDjdZ8KREqXvBEJc7Xt5bcQ0OjdgScagYTVApI/RHMiw5y4Lu0PUNPscAWiNBxbbLze7dy0muMoyLiZWJp/aZU3lAKO2iYVLbGw6IjTyuRi+4groqHSNfuLLu2e7ssBUsLW2XpKAea5o5f0Ml1FQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=bjfc7Xql; arc=none smtp.client-ip=209.85.221.43 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="bjfc7Xql" Received: by mail-wr1-f43.google.com with SMTP id ffacd0b85a97d-35f23f3da44so2294761f8f.0 for ; Tue, 11 Jun 2024 11:24:52 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1718130291; x=1718735091; darn=vger.kernel.org; h=cc:to:mime-version:content-transfer-encoding:fcc:subject:date:from :references:in-reply-to:message-id:from:to:cc:subject:date :message-id:reply-to; bh=J6phZVwBXrsTJRIGQvTlUYZADysZjBXsqztO7asPnDE=; b=bjfc7Xql3hB7VfkyaJ3hTkOA4sUP5kFvBqCMCp+lOb8rf7ICDb+ca/JMxcXuO3wJ56 iX4h0KQcjcVoKF48kvXaTk9ywy8rvqDJblAgpPY4gfrTChLnVfVwYgQG84j+N/Nk/sR7 NcpQaJT7NflqMS2nMJ+28AjyRdfYts2ar398usPnwP11q90aihe4uDTzZdEjSBcQKE7m aaL6t4uOAzgSGk+gUElkULBfHSLVGFnd6XIYkWRvK+rCxOfCzSsnQbOjKF7cOqYHeTwL KTBbDSnqIetPwO0Dk1DMPULbRQFYQxT2rvgCvERfV7fVAxLRRj3EW0Ay/1jxJ20jB06A 6Ukw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1718130291; x=1718735091; h=cc:to:mime-version:content-transfer-encoding:fcc:subject:date:from :references:in-reply-to:message-id:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=J6phZVwBXrsTJRIGQvTlUYZADysZjBXsqztO7asPnDE=; b=tIFtpE+YqYoLxDhPPoiVsahAb/Og0khYCYu2nug3fxhd0NJxwvBMNaY2GXjGI6NPG1 MZPOaKB3G2KGDwGvmrIm0Z7800bbgq7qc07wSNJoCwgRMAMTFenjdESbdw+464BeXaAy Z+1Bsuowtu/lgNBgNZRm/m3/yN8pIOe8kQX7u1IOsvLfHH5EFvA6uRixwq6m7a4Ypi7A EB5qBsUpnIKzuob/lP6kW6d0FtCke3mWRQKJMcP1XRSqGHQdJGih/J/tKSO2kCg0JE+B oPPx97Sx3dV6ra7txZ/zRRaKTyZIMj/ShruPCENWK0T71GIRFwdJ94iadoiZZrcUEnSq zN0A== X-Gm-Message-State: AOJu0YwxFLctBWni/nWOmqIETs//T6S/5xEEuYBC9eqpYIaTO14wqyvf wChLzByrfDc8MsfSj9BQLhZpM6CLDlUYujM9O8MQGYYiRlvfWxZKLi4z9Q== X-Google-Smtp-Source: AGHT+IFw6B3BwnI9QWggsAOd03gq8cq3qNJ0SeduRlzfWoO0Dj5mIxLbAq72LqtOiytICEfYukEsWA== X-Received: by 2002:a05:6000:1449:b0:35f:23f3:e95 with SMTP id ffacd0b85a97d-35f23f30fd6mr5582689f8f.35.1718130290780; Tue, 11 Jun 2024 11:24:50 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-35f2774bb64sm4946140f8f.103.2024.06.11.11.24.50 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 11 Jun 2024 11:24:50 -0700 (PDT) Message-Id: <074dc98acc79e08d07cf4f5c8105b872ec57980c.1718130288.git.gitgitgadget@gmail.com> In-Reply-To: References: Date: Tue, 11 Jun 2024 18:24:33 +0000 Subject: [PATCH 01/16] mktree: use OPT_BOOL Fcc: Sent Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 To: git@vger.kernel.org Cc: Victoria Dye , Victoria Dye From: Victoria Dye From: Victoria Dye Replace 'OPT_SET_INT' with 'OPT_BOOL' for the options '--missing' and '--batch'. The use of 'OPT_SET_INT' in these options is identical to 'OPT_BOOL', but 'OPT_BOOL' provides slightly simpler syntax. Signed-off-by: Victoria Dye --- builtin/mktree.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/builtin/mktree.c b/builtin/mktree.c index 9a22d4e2773..8b19d440747 100644 --- a/builtin/mktree.c +++ b/builtin/mktree.c @@ -162,8 +162,8 @@ int cmd_mktree(int ac, const char **av, const char *prefix) const struct option option[] = { OPT_BOOL('z', NULL, &nul_term_line, N_("input is NUL terminated")), - OPT_SET_INT( 0 , "missing", &allow_missing, N_("allow missing objects"), 1), - OPT_SET_INT( 0 , "batch", &is_batch_mode, N_("allow creation of more than one tree"), 1), + OPT_BOOL(0, "missing", &allow_missing, N_("allow missing objects")), + OPT_BOOL(0, "batch", &is_batch_mode, N_("allow creation of more than one tree")), OPT_END() }; From patchwork Tue Jun 11 18:24:34 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Victoria Dye X-Patchwork-Id: 13694104 Received: from mail-lj1-f175.google.com (mail-lj1-f175.google.com [209.85.208.175]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 618272746D for ; Tue, 11 Jun 2024 18:24:54 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.208.175 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1718130296; cv=none; b=hUY+r189WTUD2ptJOHE8PJxNizAom5uHHMGy9iTL5GFJX3x4FCuOzdtA/fqJ8e0lonhzNgVMqmeeG/pYxv88JwQtcW+lMQF+0d84Zt4DrHyMC4B2EBpofhpsL2JuwITUac6D6l3L9oyyZPy5nkVNeBnetcc9MMSmL6swJg1sNeM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1718130296; c=relaxed/simple; bh=Y3uGIenf3VzNzcILgOt57Fdj/xN4aa5LA6kSN4497Ug=; h=Message-Id:In-Reply-To:References:From:Date:Subject:Content-Type: MIME-Version:To:Cc; b=sNbRTxslLKq7Z17CYI4e47n+TQnIhEg3zK0aGIxi6AhhHdzy2D9y33r0Izvy+FejoY8C+JaHnNlt++dbLWic0s57k8mnySaQ7clGk/thH4Xy6QxD7fLpsICypg1ha8cn4FdpJJkexgYa8n7EvfuHGs9fDt75gj5Q685rsDeFC/k= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=jBMpXOYG; arc=none smtp.client-ip=209.85.208.175 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="jBMpXOYG" Received: by mail-lj1-f175.google.com with SMTP id 38308e7fff4ca-2ead2c6b50bso64444081fa.0 for ; Tue, 11 Jun 2024 11:24:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1718130292; x=1718735092; darn=vger.kernel.org; h=cc:to:mime-version:content-transfer-encoding:fcc:subject:date:from :references:in-reply-to:message-id:from:to:cc:subject:date :message-id:reply-to; bh=Urh7rNdpn2o/o6whqr985oqkaAGHqzZf69s+OZ65PsE=; b=jBMpXOYGyaxiCKu4F+h6nmnBOw2lbi+sugBIWRDqHC4Tj7wZLbZ5XHkH5B63wXbPff zb+EaxoFGYS6q5doJSvOvGTv1BzCDJiPhCvBemt7R79lEyOBXfi9LDTLJUnn+S+/VZlm wVV1ZRQRDdgH80RbqU85HmeSPmX0yhO+l+a4Otbg3TJFqCdXU5j1FSPIq4RrFgXHlmYN 1I3GRbd8LgsQsoPXM3tqzrkLfOAzg4WQNj66sqZsq+aE3TFsiuZsFpfHBP129AN5U18a ART1OetZuvkvKNgBZniELYfJzeb/RInesOZz07/m0At1V2TSOYEZYhiPeViOwoTynU2L aYlg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1718130292; x=1718735092; h=cc:to:mime-version:content-transfer-encoding:fcc:subject:date:from :references:in-reply-to:message-id:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Urh7rNdpn2o/o6whqr985oqkaAGHqzZf69s+OZ65PsE=; b=YnYCiR0FHfBjjZ5LRm+3fKXXDGQ2MND8yrrNjYaiTycqu+PfqDjNi7+sJ4cBqESlRq WQ1lGzew0CQzdQnPL7nVGrqjfkT0Huo80WU2WHFxjQOSES7pWiTRGqWyle/77m4J6XHg /uvZ3i0XfQZ9qSg8ZSmwFdMC1QA4nzecjjb7wgKs3vjhN9ymLhvY1p65lmgBQjD71S8A 4toE7CmhbPwCy3G1Pke9bEcIGFdKgAjKy0+5wi12ikOTEukrYViuijWVvHsmB/UpkynK eij93tjyDvf4n1umka+KPXBIMJ4llPz2Y9jD2QCnIRGXtDLRhiL/3A44jxvj+QCRq6ai aO0w== X-Gm-Message-State: AOJu0YwpqBnH/KbJC27syB1HKVOzJkzQVrel5dv/DQBVJnxSyOJXy+yz e+c+CSv40zYvvSZI9tIl7o3ay00AJlSph5apugug5SGznuIfFfjTDFtZig== X-Google-Smtp-Source: AGHT+IGtTbd6ANRM+Ti0MF8j6dXWqQ3Pjyax1fgrjO+XE/QlbJJWwZ58lX0beMXqjfTpE6xOdnGhaA== X-Received: by 2002:a2e:3615:0:b0:2eb:e9c0:ab69 with SMTP id 38308e7fff4ca-2ebe9c0ac3bmr33700901fa.39.1718130291889; Tue, 11 Jun 2024 11:24:51 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-421f053bf89sm67305055e9.26.2024.06.11.11.24.51 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 11 Jun 2024 11:24:51 -0700 (PDT) Message-Id: <4558f35e7bf9a1594510951ee54252069bdcfc5b.1718130288.git.gitgitgadget@gmail.com> In-Reply-To: References: Date: Tue, 11 Jun 2024 18:24:34 +0000 Subject: [PATCH 02/16] mktree: rename treeent to tree_entry Fcc: Sent Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 To: git@vger.kernel.org Cc: Victoria Dye , Victoria Dye From: Victoria Dye From: Victoria Dye Rename the type for better readability, clearly specifying "entry" (instead of the "ent" abbreviation) and separating "tree" from "entry". Signed-off-by: Victoria Dye --- builtin/mktree.c | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/builtin/mktree.c b/builtin/mktree.c index 8b19d440747..c02feb06aff 100644 --- a/builtin/mktree.c +++ b/builtin/mktree.c @@ -12,7 +12,7 @@ #include "parse-options.h" #include "object-store-ll.h" -static struct treeent { +static struct tree_entry { unsigned mode; struct object_id oid; int len; @@ -22,7 +22,7 @@ static int alloc, used; static void append_to_tree(unsigned mode, struct object_id *oid, char *path) { - struct treeent *ent; + struct tree_entry *ent; size_t len = strlen(path); if (strchr(path, '/')) die("path %s contains slash", path); @@ -38,8 +38,8 @@ static void append_to_tree(unsigned mode, struct object_id *oid, char *path) static int ent_compare(const void *a_, const void *b_) { - struct treeent *a = *(struct treeent **)a_; - struct treeent *b = *(struct treeent **)b_; + struct tree_entry *a = *(struct tree_entry **)a_; + struct tree_entry *b = *(struct tree_entry **)b_; return base_name_compare(a->name, a->len, a->mode, b->name, b->len, b->mode); } @@ -56,7 +56,7 @@ static void write_tree(struct object_id *oid) strbuf_init(&buf, size); for (i = 0; i < used; i++) { - struct treeent *ent = entries[i]; + struct tree_entry *ent = entries[i]; strbuf_addf(&buf, "%o %s%c", ent->mode, ent->name, '\0'); strbuf_add(&buf, ent->oid.hash, the_hash_algo->rawsz); } From patchwork Tue Jun 11 18:24:35 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Victoria Dye X-Patchwork-Id: 13694105 Received: from mail-wm1-f43.google.com (mail-wm1-f43.google.com [209.85.128.43]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id CDB776EB7C for ; Tue, 11 Jun 2024 18:24:54 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.43 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1718130297; cv=none; b=DsDYNi4hr+vTGIkehp2Yr+nJA3bqeGTToDUk0uoUcnkBdE8T+b3UHe9PqgV5igNMXxNqlrNjbJZbtWPFd6O7G4NSyXQaR5EtTtVi0nDZobnNOv9tMOv9KOGJ+d0vAG4AXPjPT1Y/SKnyABMQQFS1375HtpCF+C82r740x1rHMI4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1718130297; c=relaxed/simple; bh=giZOSQaaicN2icYxYpzSBgby9eHhjxzSw2wZ5PkEF5c=; h=Message-Id:In-Reply-To:References:From:Date:Subject:Content-Type: MIME-Version:To:Cc; b=lpplzzG9dhE8IODAMFn5nVmojTyhkyScits/n2T+37RFokw64T8FkQBWFno3uQIpSyDZ4Tr8UFY0eIE70rQQWeREkeavWyVNgyZiCfctEiIV9osHdj+PdDul86tt+uOw2ZQj7BQHXEA+T54MqriuutS8RGKRDwb6xEpsBbpfZdg= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=eiuQrGiK; arc=none smtp.client-ip=209.85.128.43 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="eiuQrGiK" Received: by mail-wm1-f43.google.com with SMTP id 5b1f17b1804b1-42198492353so11139605e9.1 for ; Tue, 11 Jun 2024 11:24:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1718130293; x=1718735093; darn=vger.kernel.org; h=cc:to:mime-version:content-transfer-encoding:fcc:subject:date:from :references:in-reply-to:message-id:from:to:cc:subject:date :message-id:reply-to; bh=/e400LHKdvhOuQ+OVWma9RT1Sfyor6/Vi4GK8onwiwk=; b=eiuQrGiKUMh1zx1ELFulH0rrb0NEjZdmAOd8c6aTHDc8goZqhF58dNoeVq/dH0J3qY 8bnKc+5APAsaroJTu/AKlZ7lIn4NNLi3+zlewnOhgFFpAVZNZM5hA7uREkWvlfLflYCo +0lGtvi6VGptMxddm3/W/OxwGZcU01j96e0fLeqPijoHsJH+UMWaCjwvq+bTtm5g2vrq 2mpAK/5AQjSGuMKSYa42p9F/16BMY3RR8U5FKprhMYSg4e5ySDuO0j4dLSX2KY0y33zI IdelIvj3JvbAIJTz8LQk2GXLNKEuGZcQsRi//E9gD0aZL8A8vKOfG2+wQtrsXrJ6OCi5 MT5Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1718130293; x=1718735093; h=cc:to:mime-version:content-transfer-encoding:fcc:subject:date:from :references:in-reply-to:message-id:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=/e400LHKdvhOuQ+OVWma9RT1Sfyor6/Vi4GK8onwiwk=; b=d4cT1CLsN/u9haEX1HSfN89UK5h85+13DaVkQo1JIRdTPFywCZab2upjlIAe5jOgP8 Ds0P9A6l46fNXWEYD9IZDflQg0GYLEJwxv6BdIp5Oe1fl5382bpwEkNFa8pzwxhJEuEx nuwfXSQwRZiisitEZXC/J1PSdcceuUTyXBZugyrEcSVSe3PXEDilmu11JP/ZQtB3GlXV mloAD28oiD7wRr2lnk3/3MhmTb6u7aNOByEqmuz3FJmoLjyBoGN5WPGz5ebiEj/deNYL aVByGL3aLDHcHUs1TX80y209OQZZeCKxh+4/ND/6xB5/gM2k2G21aN6S0APSFNDQnyjy mvvg== X-Gm-Message-State: AOJu0Yxngq5Q9ttd57WJbpR/Coa7l15ygrJ4tf9WqoNMWMm1IA6m63zt f1Kb86ZSkioZZcl+Tbghubqgn4s0p7di/sxddl9ZZMe8HI8Nt4nWHYfWGQ== X-Google-Smtp-Source: AGHT+IGJYLXc+7S0WuObEK/42qJtKzvW/wPm51eHTuaL3CN7Z60Z7/eEzH51l8eRRvHmu6hu7+u/qQ== X-Received: by 2002:a05:600c:3c83:b0:41c:13f6:206d with SMTP id 5b1f17b1804b1-42164a2e9d4mr117116145e9.25.1718130292609; Tue, 11 Jun 2024 11:24:52 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-422048a40a3sm63984315e9.33.2024.06.11.11.24.52 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 11 Jun 2024 11:24:52 -0700 (PDT) Message-Id: <5ade145352f44b431c16a2ec29cd87de489e8032.1718130288.git.gitgitgadget@gmail.com> In-Reply-To: References: Date: Tue, 11 Jun 2024 18:24:35 +0000 Subject: [PATCH 03/16] mktree: use non-static tree_entry array Fcc: Sent Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 To: git@vger.kernel.org Cc: Victoria Dye , Victoria Dye From: Victoria Dye From: Victoria Dye Replace the static 'struct tree_entry **entries' with a non-static 'struct tree_entry_array' instance. In later commits, we'll want to be able to create additional 'struct tree_entry_array' instances utilizing common functionality (create, push, clear, free). To avoid code duplication, create the 'struct tree_entry_array' type and add functions that perform those basic operations. Signed-off-by: Victoria Dye --- builtin/mktree.c | 67 +++++++++++++++++++++++++++++++++--------------- 1 file changed, 47 insertions(+), 20 deletions(-) diff --git a/builtin/mktree.c b/builtin/mktree.c index c02feb06aff..15bd908702a 100644 --- a/builtin/mktree.c +++ b/builtin/mktree.c @@ -12,15 +12,39 @@ #include "parse-options.h" #include "object-store-ll.h" -static struct tree_entry { +struct tree_entry { unsigned mode; struct object_id oid; int len; char name[FLEX_ARRAY]; -} **entries; -static int alloc, used; +}; + +struct tree_entry_array { + size_t nr, alloc; + struct tree_entry **entries; +}; -static void append_to_tree(unsigned mode, struct object_id *oid, char *path) +static void tree_entry_array_push(struct tree_entry_array *arr, struct tree_entry *ent) +{ + ALLOC_GROW(arr->entries, arr->nr + 1, arr->alloc); + arr->entries[arr->nr++] = ent; +} + +static void clear_tree_entry_array(struct tree_entry_array *arr) +{ + for (size_t i = 0; i < arr->nr; i++) + FREE_AND_NULL(arr->entries[i]); + arr->nr = 0; +} + +static void release_tree_entry_array(struct tree_entry_array *arr) +{ + FREE_AND_NULL(arr->entries); + arr->nr = arr->alloc = 0; +} + +static void append_to_tree(unsigned mode, struct object_id *oid, const char *path, + struct tree_entry_array *arr) { struct tree_entry *ent; size_t len = strlen(path); @@ -32,8 +56,8 @@ static void append_to_tree(unsigned mode, struct object_id *oid, char *path) ent->len = len; oidcpy(&ent->oid, oid); - ALLOC_GROW(entries, used + 1, alloc); - entries[used++] = ent; + /* Append the update */ + tree_entry_array_push(arr, ent); } static int ent_compare(const void *a_, const void *b_) @@ -44,19 +68,18 @@ static int ent_compare(const void *a_, const void *b_) b->name, b->len, b->mode); } -static void write_tree(struct object_id *oid) +static void write_tree(struct tree_entry_array *arr, struct object_id *oid) { struct strbuf buf; - size_t size; - int i; + size_t size = 0; - QSORT(entries, used, ent_compare); - for (size = i = 0; i < used; i++) - size += 32 + entries[i]->len; + QSORT(arr->entries, arr->nr, ent_compare); + for (size_t i = 0; i < arr->nr; i++) + size += 32 + arr->entries[i]->len; strbuf_init(&buf, size); - for (i = 0; i < used; i++) { - struct tree_entry *ent = entries[i]; + for (size_t i = 0; i < arr->nr; i++) { + struct tree_entry *ent = arr->entries[i]; strbuf_addf(&buf, "%o %s%c", ent->mode, ent->name, '\0'); strbuf_add(&buf, ent->oid.hash, the_hash_algo->rawsz); } @@ -70,7 +93,8 @@ static const char *mktree_usage[] = { NULL }; -static void mktree_line(char *buf, int nul_term_line, int allow_missing) +static void mktree_line(char *buf, int nul_term_line, int allow_missing, + struct tree_entry_array *arr) { char *ptr, *ntr; const char *p; @@ -146,7 +170,7 @@ static void mktree_line(char *buf, int nul_term_line, int allow_missing) } } - append_to_tree(mode, &oid, path); + append_to_tree(mode, &oid, path, arr); free(to_free); } @@ -158,6 +182,7 @@ int cmd_mktree(int ac, const char **av, const char *prefix) int allow_missing = 0; int is_batch_mode = 0; int got_eof = 0; + struct tree_entry_array arr = { 0 }; strbuf_getline_fn getline_fn; const struct option option[] = { @@ -182,9 +207,9 @@ int cmd_mktree(int ac, const char **av, const char *prefix) break; die("input format error: (blank line only valid in batch mode)"); } - mktree_line(sb.buf, nul_term_line, allow_missing); + mktree_line(sb.buf, nul_term_line, allow_missing, &arr); } - if (is_batch_mode && got_eof && used < 1) { + if (is_batch_mode && got_eof && arr.nr < 1) { /* * Execution gets here if the last tree entry is terminated with a * new-line. The final new-line has been made optional to be @@ -192,12 +217,14 @@ int cmd_mktree(int ac, const char **av, const char *prefix) */ ; /* skip creating an empty tree */ } else { - write_tree(&oid); + write_tree(&arr, &oid); puts(oid_to_hex(&oid)); fflush(stdout); } - used=0; /* reset tree entry buffer for re-use in batch mode */ + clear_tree_entry_array(&arr); /* reset tree entry buffer for re-use in batch mode */ } + + release_tree_entry_array(&arr); strbuf_release(&sb); return 0; } From patchwork Tue Jun 11 18:24:36 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Victoria Dye X-Patchwork-Id: 13694106 Received: from mail-lj1-f177.google.com (mail-lj1-f177.google.com [209.85.208.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id BCF3C7441E for ; Tue, 11 Jun 2024 18:24:55 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.208.177 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1718130297; cv=none; b=O98Z9TubZha2nfNuCJS1LYOtkepwYXpiq9zmFnadLQ9wOi9F+3jAJo1EwXvHB6ZJHorMX1mN/e0kPE3wYuBbCQx/cDC4i3v7epCofNJQkZ4pGeFvU7uVW3TrnQ3XzzzeGQlPr3PYMF+M0YIvtqzo8oRkxjFA/QoqkRc+oPZaM3Q= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1718130297; c=relaxed/simple; bh=kWwvRTjg9Hemaz0ymKYMG4JoE9jb1NA6f1/ya6Sp99s=; h=Message-Id:In-Reply-To:References:From:Date:Subject:Content-Type: MIME-Version:To:Cc; b=b2TBoH5r59yKxvvJG9Ka8C6o+0IV9olyTRzcGo9QqJ2kr7s2VcGcPLtCn9oDlZ2aeScP+g2zGSs8Fk3BVu/wmTfH9eSw4VOAFDN7/8UM0xGJ521aaJgg5HnPHTWxJbT2rAWJBYgEE9R9TXzssTCTPV6YeG83OMCyBY7QJkk3euw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=Na5E3rWU; arc=none smtp.client-ip=209.85.208.177 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="Na5E3rWU" Received: by mail-lj1-f177.google.com with SMTP id 38308e7fff4ca-2ebed33cbadso19634941fa.0 for ; Tue, 11 Jun 2024 11:24:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1718130293; x=1718735093; darn=vger.kernel.org; h=cc:to:mime-version:content-transfer-encoding:fcc:subject:date:from :references:in-reply-to:message-id:from:to:cc:subject:date :message-id:reply-to; bh=MKqRbGm/RW+V6fq9cVwuHYuiNBW4lFAV7W6d0xw6R6M=; b=Na5E3rWUWkKavbNVE6aKfC3eIQDsnbgsNU7NttISmqUNyyK2SOQ6xwBwb9wRh1b9m3 L3WVmgsqO2euZ4F2+TfhnbHoAzIbM7Hp4rRGW1pVTQ+PsfhRsySJ2MHVoxQ9gD+GG/jv Iy2YBXZLxx1nD1Qd3XVrBmPARQN35xPuz5A2pm9i2Fz/B+KaskZTpPBjfQjBC+s/F/ox GaNprYWC6hoKExDmwWLMFXSw1VDT3XAk+rBdz8IeHbMQcFUY/vIWBTl7RRtoFFwshxHF +R9rwwNwFbMKxUxBCWO86SOeKzpkTKgPrAfGCxdSubkFuHU1pHii0j2r/xbsr8JAzWCQ Q9hA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1718130293; x=1718735093; h=cc:to:mime-version:content-transfer-encoding:fcc:subject:date:from :references:in-reply-to:message-id:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=MKqRbGm/RW+V6fq9cVwuHYuiNBW4lFAV7W6d0xw6R6M=; b=XOSzhVj+8n4j52aA/Dj/GkHVb+RVQDOFU0vq8qqU0mVhFKmHVoYd50wwizE1hmILjG WQPDcD/YyF6+SNOush2btzCHko4EBtJpbJpw+U2TLiMSnmYU6fm0duUGjB+Njzooahxl h18o7zBsSvMkbFZt3QIQwuHr8tBPfmD7a2QMkuQ/D+0uWndCqeP0PZ52IrvwbCoee+gr 8shWQ1spAozjIR41Lys4Xenls0DVIrfnnmmRaicMH+x7ofq1mPpnCg4o5qwfS4dH7sO+ j6FfVHdpHiDkrpk48kadC5kZRNazeGXzLMpGcJKfVNI+CmOrydND7BgXNs28sRX47hQh pZZg== X-Gm-Message-State: AOJu0Yx2StNFypNaho6sN8OD3M2O45mvWZkCuDUM6uzFzjj3hWRhKlMG 2jZkFx3206ug5K5Y9uFBz/Wf6435p9z1tFy5sAO0EnAMFZp7KThOGRH0Bg== X-Google-Smtp-Source: AGHT+IF2AFmN76jNhi6Rdm+FY4mbp8+vfH7a8jzqRUvbrxE45X6ZDhNvUz+49NtmGjgTqHfK3QU+8g== X-Received: by 2002:a2e:9096:0:b0:2eb:1de9:bede with SMTP id 38308e7fff4ca-2eb1de9c5e2mr52514251fa.51.1718130293261; Tue, 11 Jun 2024 11:24:53 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-422760ce994sm12242195e9.13.2024.06.11.11.24.52 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 11 Jun 2024 11:24:52 -0700 (PDT) Message-Id: <9d0689e9c285b375b0067760929011038c085d65.1718130288.git.gitgitgadget@gmail.com> In-Reply-To: References: Date: Tue, 11 Jun 2024 18:24:36 +0000 Subject: [PATCH 04/16] update-index: generalize 'read_index_info' Fcc: Sent Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 To: git@vger.kernel.org Cc: Victoria Dye , Victoria Dye From: Victoria Dye From: Victoria Dye Move 'read_index_info()' into a new header 'index-info.h' and generalize the function to call a provided callback for each parsed line. Update 'update-index.c' to use this generalized 'read_index_info()', adding the callback 'apply_index_info()' to verify the parsed line and update the index according to its contents. The input parsing done by 'read_index_info()' is similar to, but more flexible than, the parsing done in 'mktree' by 'mktree_line()' (handling not only 'git ls-tree' output but also the outputs of 'git apply --index-info' and 'git ls-files --stage' outputs). To make 'mktree' more flexible, a later patch will replace mktree's custom parsing with 'read_index_info()'. Signed-off-by: Victoria Dye --- Makefile | 1 + builtin/update-index.c | 116 ++++++++-------------------------- index-info.c | 91 ++++++++++++++++++++++++++ index-info.h | 11 ++++ t/t2107-update-index-basic.sh | 27 ++++++++ 5 files changed, 155 insertions(+), 91 deletions(-) create mode 100644 index-info.c create mode 100644 index-info.h diff --git a/Makefile b/Makefile index 2f5f16847ae..db9604e59c3 100644 --- a/Makefile +++ b/Makefile @@ -1037,6 +1037,7 @@ LIB_OBJS += hex.o LIB_OBJS += hex-ll.o LIB_OBJS += hook.o LIB_OBJS += ident.o +LIB_OBJS += index-info.o LIB_OBJS += json-writer.o LIB_OBJS += kwset.o LIB_OBJS += levenshtein.o diff --git a/builtin/update-index.c b/builtin/update-index.c index d343416ae26..77df380cb54 100644 --- a/builtin/update-index.c +++ b/builtin/update-index.c @@ -11,6 +11,7 @@ #include "gettext.h" #include "hash.h" #include "hex.h" +#include "index-info.h" #include "lockfile.h" #include "quote.h" #include "cache-tree.h" @@ -509,100 +510,29 @@ static void update_one(const char *path) report("add '%s'", path); } -static void read_index_info(int nul_term_line) +static int apply_index_info(unsigned int mode, struct object_id *oid, int stage, + const char *path_name, void *cbdata UNUSED) { - const int hexsz = the_hash_algo->hexsz; - struct strbuf buf = STRBUF_INIT; - struct strbuf uq = STRBUF_INIT; - strbuf_getline_fn getline_fn; + if (!verify_path(path_name, mode)) { + fprintf(stderr, "Ignoring path %s\n", path_name); + return 0; + } - getline_fn = nul_term_line ? strbuf_getline_nul : strbuf_getline_lf; - while (getline_fn(&buf, stdin) != EOF) { - char *ptr, *tab; - char *path_name; - struct object_id oid; - unsigned int mode; - unsigned long ul; - int stage; - - /* This reads lines formatted in one of three formats: - * - * (1) mode SP sha1 TAB path - * The first format is what "git apply --index-info" - * reports, and used to reconstruct a partial tree - * that is used for phony merge base tree when falling - * back on 3-way merge. - * - * (2) mode SP type SP sha1 TAB path - * The second format is to stuff "git ls-tree" output - * into the index file. - * - * (3) mode SP sha1 SP stage TAB path - * This format is to put higher order stages into the - * index file and matches "git ls-files --stage" output. + if (!mode) { + /* mode == 0 means there is no such path -- remove */ + if (remove_file_from_index(the_repository->index, path_name)) + die("git update-index: unable to remove %s", path_name); + } + else { + /* mode ' ' sha1 '\t' name + * ptr[-1] points at tab, + * ptr[-41] is at the beginning of sha1 */ - errno = 0; - ul = strtoul(buf.buf, &ptr, 8); - if (ptr == buf.buf || *ptr != ' ' - || errno || (unsigned int) ul != ul) - goto bad_line; - mode = ul; - - tab = strchr(ptr, '\t'); - if (!tab || tab - ptr < hexsz + 1) - goto bad_line; - - if (tab[-2] == ' ' && '0' <= tab[-1] && tab[-1] <= '3') { - stage = tab[-1] - '0'; - ptr = tab + 1; /* point at the head of path */ - tab = tab - 2; /* point at tail of sha1 */ - } - else { - stage = 0; - ptr = tab + 1; /* point at the head of path */ - } - - if (get_oid_hex(tab - hexsz, &oid) || - tab[-(hexsz + 1)] != ' ') - goto bad_line; - - path_name = ptr; - if (!nul_term_line && path_name[0] == '"') { - strbuf_reset(&uq); - if (unquote_c_style(&uq, path_name, NULL)) { - die("git update-index: bad quoting of path name"); - } - path_name = uq.buf; - } - - if (!verify_path(path_name, mode)) { - fprintf(stderr, "Ignoring path %s\n", path_name); - continue; - } - - if (!mode) { - /* mode == 0 means there is no such path -- remove */ - if (remove_file_from_index(the_repository->index, path_name)) - die("git update-index: unable to remove %s", - ptr); - } - else { - /* mode ' ' sha1 '\t' name - * ptr[-1] points at tab, - * ptr[-41] is at the beginning of sha1 - */ - ptr[-(hexsz + 2)] = ptr[-1] = 0; - if (add_cacheinfo(mode, &oid, path_name, stage)) - die("git update-index: unable to update %s", - path_name); - } - continue; - - bad_line: - die("malformed index info %s", buf.buf); + if (add_cacheinfo(mode, oid, path_name, stage)) + die("git update-index: unable to update %s", path_name); } - strbuf_release(&buf); - strbuf_release(&uq); + + return 0; } static const char * const update_index_usage[] = { @@ -849,6 +779,7 @@ static enum parse_opt_result stdin_cacheinfo_callback( const char *arg, int unset) { int *nul_term_line = opt->value; + int ret; BUG_ON_OPT_NEG(unset); BUG_ON_OPT_ARG(arg); @@ -856,7 +787,10 @@ static enum parse_opt_result stdin_cacheinfo_callback( if (ctx->argc != 1) return error("option '%s' must be the last argument", opt->long_name); allow_add = allow_replace = allow_remove = 1; - read_index_info(*nul_term_line); + ret = read_index_info(*nul_term_line, apply_index_info, NULL); + if (ret) + return -1; + return 0; } diff --git a/index-info.c b/index-info.c new file mode 100644 index 00000000000..0b68e34c361 --- /dev/null +++ b/index-info.c @@ -0,0 +1,91 @@ +#include "git-compat-util.h" +#include "index-info.h" +#include "hash.h" +#include "hex.h" +#include "strbuf.h" +#include "quote.h" + +int read_index_info(int nul_term_line, each_index_info_fn fn, void *cbdata) +{ + const int hexsz = the_hash_algo->hexsz; + struct strbuf buf = STRBUF_INIT; + struct strbuf uq = STRBUF_INIT; + strbuf_getline_fn getline_fn; + int ret = 0; + + getline_fn = nul_term_line ? strbuf_getline_nul : strbuf_getline_lf; + while (getline_fn(&buf, stdin) != EOF) { + char *ptr, *tab; + char *path_name; + struct object_id oid; + unsigned int mode; + unsigned long ul; + int stage; + + /* This reads lines formatted in one of three formats: + * + * (1) mode SP sha1 TAB path + * The first format is what "git apply --index-info" + * reports, and used to reconstruct a partial tree + * that is used for phony merge base tree when falling + * back on 3-way merge. + * + * (2) mode SP type SP sha1 TAB path + * The second format is to stuff "git ls-tree" output + * into the index file. + * + * (3) mode SP sha1 SP stage TAB path + * This format is to put higher order stages into the + * index file and matches "git ls-files --stage" output. + */ + errno = 0; + ul = strtoul(buf.buf, &ptr, 8); + if (ptr == buf.buf || *ptr != ' ' + || errno || (unsigned int) ul != ul) + goto bad_line; + mode = ul; + + tab = strchr(ptr, '\t'); + if (!tab || tab - ptr < hexsz + 1) + goto bad_line; + + if (tab[-2] == ' ' && '0' <= tab[-1] && tab[-1] <= '3') { + stage = tab[-1] - '0'; + ptr = tab + 1; /* point at the head of path */ + tab = tab - 2; /* point at tail of sha1 */ + } else { + stage = 0; + ptr = tab + 1; /* point at the head of path */ + } + + if (get_oid_hex(tab - hexsz, &oid) || + tab[-(hexsz + 1)] != ' ') + goto bad_line; + + path_name = ptr; + if (!nul_term_line && path_name[0] == '"') { + strbuf_reset(&uq); + if (unquote_c_style(&uq, path_name, NULL)) { + ret = error("bad quoting of path name"); + break; + } + path_name = uq.buf; + } + + ret = fn(mode, &oid, stage, path_name, cbdata); + if (ret) { + ret = -1; + break; + } + + continue; + + bad_line: + ret = error("malformed input line '%s'", buf.buf); + break; + } + strbuf_release(&buf); + strbuf_release(&uq); + + return ret; +} diff --git a/index-info.h b/index-info.h new file mode 100644 index 00000000000..d650498325a --- /dev/null +++ b/index-info.h @@ -0,0 +1,11 @@ +#ifndef INDEX_INFO_H +#define INDEX_INFO_H + +#include "hash.h" + +typedef int (*each_index_info_fn)(unsigned int, struct object_id *, int, const char *, void *); + +/* Iterate over parsed index info from stdin */ +int read_index_info(int nul_term_line, each_index_info_fn fn, void *cbdata); + +#endif /* INDEX_INFO_H */ diff --git a/t/t2107-update-index-basic.sh b/t/t2107-update-index-basic.sh index cc72ead79f3..29696ade0d0 100755 --- a/t/t2107-update-index-basic.sh +++ b/t/t2107-update-index-basic.sh @@ -142,4 +142,31 @@ test_expect_success '--index-version' ' test_must_be_empty actual ' +test_expect_success '--index-info fails on malformed input' ' + # empty line + echo "" | + test_must_fail git update-index --index-info 2>err && + grep "malformed input line" err && + + # bad whitespace + printf "100644 $EMPTY_BLOB A" | + test_must_fail git update-index --index-info 2>err && + grep "malformed input line" err && + + # invalid stage value + printf "100644 $EMPTY_BLOB 5\tA" | + test_must_fail git update-index --index-info 2>err && + grep "malformed input line" err && + + # invalid OID length + printf "100755 abc123\tA" | + test_must_fail git update-index --index-info 2>err && + grep "malformed input line" err && + + # bad quoting + printf "100644 $EMPTY_BLOB\t\"A" | + test_must_fail git update-index --index-info 2>err && + grep "bad quoting of path name" err +' + test_done From patchwork Tue Jun 11 18:24:37 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Victoria Dye X-Patchwork-Id: 13694108 Received: from mail-lf1-f49.google.com (mail-lf1-f49.google.com [209.85.167.49]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E5217762C9 for ; Tue, 11 Jun 2024 18:24:57 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.167.49 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1718130299; cv=none; b=Fqh7OnQIM+ovlf8WGDAE9nslDuRyMlHn15gwuWVXxzx2fSn3crEK5RU6rJ2C1fvfJEAvbRGoSfIjCcqoOkGyaXb+jKi8GA1AV64X415iHbXCxBqlBkfiXRZJXFru8MGz+uPSeB9oc5zGWpTnss3mGucJGWPKI3nBT2gC/61nT2U= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1718130299; c=relaxed/simple; bh=6iVA7TIhgsaPJnq/fBFOf0AgwGY4pBN8PPR8dYwuQg4=; h=Message-Id:In-Reply-To:References:From:Date:Subject:Content-Type: MIME-Version:To:Cc; b=rdUvxa2Ji7jcNLVsBLWm5WWI6tYm0mQ6PZJWU7SFPN29kK+ZmlNlXcSprHPMLcMWGNbrlCSLUHHnBgYe+ecpWtrIW/s+mmMRk2lFkW7u0byC6CJt1nWer719O+q4SuW/QQItGUWMKhRdTxOl7/VTVGUiAVWGCjdzFp3MqendU8g= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=IxZNH8Oh; arc=none smtp.client-ip=209.85.167.49 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="IxZNH8Oh" Received: by mail-lf1-f49.google.com with SMTP id 2adb3069b0e04-52c8973ff29so1640732e87.0 for ; Tue, 11 Jun 2024 11:24:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1718130294; x=1718735094; darn=vger.kernel.org; h=cc:to:mime-version:content-transfer-encoding:fcc:subject:date:from :references:in-reply-to:message-id:from:to:cc:subject:date :message-id:reply-to; bh=baomfKBJJHmpTlw3XLQPe4j8XX179oT0HRdiBQrudqg=; b=IxZNH8Oh2WrzoPXfYMXPKqYqLgaOkBofMGY0yT+2ZkvpBdfseaN5lzDShGT/n9YGpD vc1/ovfP3Q3q6QbqAOU/2WEpiPfDc9FrSa1sI8qOh4mNCbLVZUrO+AO3ZIwQbY+MvPJA YqfrsibodtxWc6L2HxQFijseS5K22lMkQCYnuDEFYw02oGWipalLsJONzA1NiJkBbs2e sDhpFPWO99PpmrSt0a1vGuwAqlZD9dDEnyICVYcwvjFZmfHabUmPfjtrrf40wO2xm1dg 1u33es9touQJdIloxnxdoY+NL5xJHUkZOQk/8Okqv3DIqV6ZMIbVqYTMrxYBOeovsdqr lj3w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1718130294; x=1718735094; h=cc:to:mime-version:content-transfer-encoding:fcc:subject:date:from :references:in-reply-to:message-id:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=baomfKBJJHmpTlw3XLQPe4j8XX179oT0HRdiBQrudqg=; b=gS4s719h6werprBk20GX+HO50l/pJDBvQeapguh5DS42cCupcPrX9k6QvEoXebDTaL YXgeEAobKYuBfeePctJRsO+ogUF/D5I7KMJ+Mprw8I2oF7hi5WS9H0nSN0651aOQ0H2M 1/ZzKKwj26JXQSU3n7YhcFeC1NEIwE+AWIoOXzz6h1fQYeE7xgk5NCH1JmDVqvMf7r+U GRsQbbmyuwEVXlMhCOtOAn9soWoSKVYxETTL9H/7VNHuppeyMgR+Myx1Y2NE8jn4MPLN LM7s2EfoC4TcI4S6DKg4YWdO6tcHomXSvVPqRv2Jh3TSwuE6j3N/KGdAX7LTVi7seKv5 SIZw== X-Gm-Message-State: AOJu0YwVZVy6lkOSA2/dZwYrHnPzpnsprh1asS10sGvE8HWQ/HA+sesW dMbN6y4nztlqnPiwwVvo8a/wfgKH+Wnq9gz+e8fwLEagx+PifbvfCQc1cw== X-Google-Smtp-Source: AGHT+IHc4FThWcduIo8BH16pJkR5Ci9MVE+2gwvrWMtnMRGaXTxfRk9lDa+2vDDscfipmcBQc8dzzQ== X-Received: by 2002:ac2:5dc8:0:b0:52b:c029:1bdf with SMTP id 2adb3069b0e04-52bc0291d9amr7340138e87.24.1718130293994; Tue, 11 Jun 2024 11:24:53 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-4221defede0sm51625785e9.7.2024.06.11.11.24.53 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 11 Jun 2024 11:24:53 -0700 (PDT) Message-Id: <7e3bcc16e23c97d8a4efbb9e14b230ef9f44a1a7.1718130288.git.gitgitgadget@gmail.com> In-Reply-To: References: Date: Tue, 11 Jun 2024 18:24:37 +0000 Subject: [PATCH 05/16] index-info.c: identify empty input lines in read_index_info Fcc: Sent Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 To: git@vger.kernel.org Cc: Victoria Dye , Victoria Dye From: Victoria Dye From: Victoria Dye Update 'read_index_info()' to return INDEX_INFO_EMPTY_LINE (value 1), rather than the default error code (value -1) when the function encounters an empty line in stdin. This grants the caller the flexibility to handle such scenarios differently than a typical error. In the case of 'update-index', we'll still exit with a "malformed input line" error. However, when 'read_index_info()' is used to process the input to 'mktree' in a later patch, the empty line return value will signal a new tree in --batch mode. Signed-off-by: Victoria Dye --- builtin/update-index.c | 4 +++- index-info.c | 5 +++++ index-info.h | 2 ++ 3 files changed, 10 insertions(+), 1 deletion(-) diff --git a/builtin/update-index.c b/builtin/update-index.c index 77df380cb54..b1b334807f8 100644 --- a/builtin/update-index.c +++ b/builtin/update-index.c @@ -788,7 +788,9 @@ static enum parse_opt_result stdin_cacheinfo_callback( return error("option '%s' must be the last argument", opt->long_name); allow_add = allow_replace = allow_remove = 1; ret = read_index_info(*nul_term_line, apply_index_info, NULL); - if (ret) + if (ret == INDEX_INFO_EMPTY_LINE) + return error("malformed input line ''"); + else if (ret < 0) return -1; return 0; diff --git a/index-info.c b/index-info.c index 0b68e34c361..735cbf1f476 100644 --- a/index-info.c +++ b/index-info.c @@ -22,6 +22,11 @@ int read_index_info(int nul_term_line, each_index_info_fn fn, void *cbdata) unsigned long ul; int stage; + if (!buf.len) { + ret = INDEX_INFO_EMPTY_LINE; + break; + } + /* This reads lines formatted in one of three formats: * * (1) mode SP sha1 TAB path diff --git a/index-info.h b/index-info.h index d650498325a..1884972021d 100644 --- a/index-info.h +++ b/index-info.h @@ -5,6 +5,8 @@ typedef int (*each_index_info_fn)(unsigned int, struct object_id *, int, const char *, void *); +#define INDEX_INFO_EMPTY_LINE 1 + /* Iterate over parsed index info from stdin */ int read_index_info(int nul_term_line, each_index_info_fn fn, void *cbdata); From patchwork Tue Jun 11 18:24:38 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Victoria Dye X-Patchwork-Id: 13694107 Received: from mail-wr1-f45.google.com (mail-wr1-f45.google.com [209.85.221.45]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 0A81F74E2E for ; Tue, 11 Jun 2024 18:24:56 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.221.45 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1718130298; cv=none; b=VsN0vzqOCRdRbDhTbJ0cPo8fVoOU84LsmD2PqmS3y3RcqUKZi4DCpD8v6cm71uCz7sJigwgYw6N0yREk5jXwWcaYHeFLEZwT6ZtSK8XEB6ZbDvjMp9n9RwG6yzjsSnXlAOGSRkKSyzn0araqILtr0qNS8xb8yotjty5Jp6QRxdM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1718130298; c=relaxed/simple; bh=kEfIRVZqmQ7lQD9kQ6ckbaiUmjNHuzqSf9NtVk8uWsA=; h=Message-Id:In-Reply-To:References:From:Date:Subject:Content-Type: MIME-Version:To:Cc; b=kuXJLfHvHljICB8oAKzp8FnGoW/+06WC+tJ4xGx8fYtKNKZMldvVuNsIoG8YgVTxTdneU5DPQgTqlaSa22egoYtMtN3t9DpGojHKSgL1vm+1+Yb8CklAhR2oNrecYMPYt3GqXx5n3XI5HZdE0fklfq+QQBOhQIDYgak68upKgaI= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=lWKoTwCk; arc=none smtp.client-ip=209.85.221.45 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="lWKoTwCk" Received: by mail-wr1-f45.google.com with SMTP id ffacd0b85a97d-35f123bf735so991646f8f.1 for ; Tue, 11 Jun 2024 11:24:56 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1718130295; x=1718735095; darn=vger.kernel.org; h=cc:to:mime-version:content-transfer-encoding:fcc:subject:date:from :references:in-reply-to:message-id:from:to:cc:subject:date :message-id:reply-to; bh=SavWhNI94OBdgyA467GiVzYUN+BXnMl+ZcQRz1bzBfE=; b=lWKoTwCkOhyiKtuJFJn8va2dW/FDSl8lwOo7AZxoKMM+p3xP/JZrecJZ2evYJ2xziC /JZYEOJtyeEcK2IXumA3JzlzVxAHO5DHZxiQmeTRRg802Y14KwdT0rkpQ19Q7he0CR7C /LuoJMxLcqU8E6YqiU2KZlXTv15kmE8Nk/sImOKVdVYNwo5yXaDV4KG0J4Yx3RLEHy02 8bbVeU1olneXZPU37+xtt8T4IADkMuPPQELFcmsjRZVWuY8tQYrBV2jI69E/9lnuYm5J imb8jopxcODvwc6HAoi/OqHLfoTWWNSeILznZzNiG3rXe9J7fDFuuo4/aCIi5zN8DtsB 6gzw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1718130295; x=1718735095; h=cc:to:mime-version:content-transfer-encoding:fcc:subject:date:from :references:in-reply-to:message-id:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=SavWhNI94OBdgyA467GiVzYUN+BXnMl+ZcQRz1bzBfE=; b=VhG1l6sQ1KyqpdNYKVAcA6uSGFfaQFOugiDvnzXnLU45k3WqaGFE8Yb0ZIkOaBi/4y QuAkdV0FKBclm+hTzgD+/YZ+7eTarQ8AI79Wln4dgQDA/T6HwULB/k4hoy+nAPPoe509 fA53IN8ZdHNAVrevoJgaq+WLQz4T5ZP8aPqwhisWYOlSrRUgF+pwOG7+W82XtUuJii67 RUWEeDbuReq+/NbdN15BxYxE4G7zHf5XnSnAtKQCVsgkTCrmE91Anwvd/oDyiEXRwE2F EFMfuNswWEZwJyQuDpoTSv0xS0uACcVq2b0vsQw+O64bAoPUAv9P5rxjJ4FUyWgCDhtX VjFg== X-Gm-Message-State: AOJu0YzKumQBKNEk3RN+cGuacWyNnk61XKWQ5VulIkGZWnWW01klWqJN XMAtvSlN4XCvA81UReWJw9VUMq2lovOKyp7QLQqyY6K9rUl2Uh93PYPKuw== X-Google-Smtp-Source: AGHT+IEYk3LsqNrF+/ysSXYFf4WB/q5xwoGKAHiyEi0AXRJ6szkdCGODBlXdW6tXy7TlF6Hl++BsiA== X-Received: by 2002:a5d:618c:0:b0:35f:29da:2345 with SMTP id ffacd0b85a97d-35f29da23e0mr3755340f8f.16.1718130294741; Tue, 11 Jun 2024 11:24:54 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-35f23df9f3csm6347945f8f.76.2024.06.11.11.24.54 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 11 Jun 2024 11:24:54 -0700 (PDT) Message-Id: In-Reply-To: References: Date: Tue, 11 Jun 2024 18:24:38 +0000 Subject: [PATCH 06/16] index-info.c: parse object type in provided in read_index_info Fcc: Sent Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 To: git@vger.kernel.org Cc: Victoria Dye , Victoria Dye From: Victoria Dye From: Victoria Dye If the object type (e.g. "blob", "tree") is identified on a stdin line read by 'read_index_info()' (i.e. on lines formatted like the output of 'git ls-tree'), parse it into an 'enum object_type' and provide it to the 'read_index_info()' callback as an argument. If the type is not provided, pass 'OBJ_NONE' instead. If the object type is invalid, return an error. The goal of this change is to allow for more thorough validation of the provided object type (e.g. against the provided mode) in 'mktree' once 'mktree_line' is replaced with 'read_index_info()'. Note, though, that this change also strengthens the validation done by 'update-index', since invalid type names now trigger an error. Signed-off-by: Victoria Dye --- builtin/update-index.c | 3 ++- index-info.c | 16 ++++++++++++---- index-info.h | 3 ++- t/t2107-update-index-basic.sh | 5 +++++ 4 files changed, 21 insertions(+), 6 deletions(-) diff --git a/builtin/update-index.c b/builtin/update-index.c index b1b334807f8..8882433b644 100644 --- a/builtin/update-index.c +++ b/builtin/update-index.c @@ -510,7 +510,8 @@ static void update_one(const char *path) report("add '%s'", path); } -static int apply_index_info(unsigned int mode, struct object_id *oid, int stage, +static int apply_index_info(unsigned int mode, struct object_id *oid, + enum object_type obj_type UNUSED, int stage, const char *path_name, void *cbdata UNUSED) { if (!verify_path(path_name, mode)) { diff --git a/index-info.c b/index-info.c index 735cbf1f476..5d61e61e28f 100644 --- a/index-info.c +++ b/index-info.c @@ -18,6 +18,7 @@ int read_index_info(int nul_term_line, each_index_info_fn fn, void *cbdata) char *ptr, *tab; char *path_name; struct object_id oid; + enum object_type obj_type = OBJ_NONE; unsigned int mode; unsigned long ul; int stage; @@ -56,18 +57,17 @@ int read_index_info(int nul_term_line, each_index_info_fn fn, void *cbdata) if (tab[-2] == ' ' && '0' <= tab[-1] && tab[-1] <= '3') { stage = tab[-1] - '0'; - ptr = tab + 1; /* point at the head of path */ + path_name = tab + 1; /* point at the head of path */ tab = tab - 2; /* point at tail of sha1 */ } else { stage = 0; - ptr = tab + 1; /* point at the head of path */ + path_name = tab + 1; /* point at the head of path */ } if (get_oid_hex(tab - hexsz, &oid) || tab[-(hexsz + 1)] != ' ') goto bad_line; - path_name = ptr; if (!nul_term_line && path_name[0] == '"') { strbuf_reset(&uq); if (unquote_c_style(&uq, path_name, NULL)) { @@ -77,7 +77,15 @@ int read_index_info(int nul_term_line, each_index_info_fn fn, void *cbdata) path_name = uq.buf; } - ret = fn(mode, &oid, stage, path_name, cbdata); + /* Get the type, if provided */ + if (tab - hexsz - 1 > ptr + 1) { + if (*(tab - hexsz - 1) != ' ') + goto bad_line; + *(tab - hexsz - 1) = '\0'; + obj_type = type_from_string(ptr + 1); + } + + ret = fn(mode, &oid, obj_type, stage, path_name, cbdata); if (ret) { ret = -1; break; diff --git a/index-info.h b/index-info.h index 1884972021d..767cf304213 100644 --- a/index-info.h +++ b/index-info.h @@ -2,8 +2,9 @@ #define INDEX_INFO_H #include "hash.h" +#include "object.h" -typedef int (*each_index_info_fn)(unsigned int, struct object_id *, int, const char *, void *); +typedef int (*each_index_info_fn)(unsigned int, struct object_id *, enum object_type, int, const char *, void *); #define INDEX_INFO_EMPTY_LINE 1 diff --git a/t/t2107-update-index-basic.sh b/t/t2107-update-index-basic.sh index 29696ade0d0..9c19d24cd4a 100755 --- a/t/t2107-update-index-basic.sh +++ b/t/t2107-update-index-basic.sh @@ -153,6 +153,11 @@ test_expect_success '--index-info fails on malformed input' ' test_must_fail git update-index --index-info 2>err && grep "malformed input line" err && + # invalid type + printf "100644 bad $EMPTY_BLOB\tA" | + test_must_fail git update-index --index-info 2>err && + grep "invalid object type" err && + # invalid stage value printf "100644 $EMPTY_BLOB 5\tA" | test_must_fail git update-index --index-info 2>err && From patchwork Tue Jun 11 18:24:39 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Victoria Dye X-Patchwork-Id: 13694110 Received: from mail-wr1-f52.google.com (mail-wr1-f52.google.com [209.85.221.52]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 977967581F for ; Tue, 11 Jun 2024 18:24:57 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.221.52 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1718130301; cv=none; b=Yw0xjWB4czS3RJkW9Q+vzdYEVWv7GT2/u5PDFBARKws6nuWzyoDmV1OFxn7JG4eAAlbtSrmsNs0uxg8vidbeHydmmNDknawBkNVGOCQq4a/FSoys5dTicmEnnlmeUtIkP362awj4Un7N3MyAetLwky77yLtJpVrkhO7h6ZhqEw4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1718130301; c=relaxed/simple; bh=bCBK4+Mj0szw2NZ/CD7kOc6CzbtdIpdljoCW0GoTyMg=; h=Message-Id:In-Reply-To:References:From:Date:Subject:Content-Type: MIME-Version:To:Cc; b=KnDrVlg6HifmM9oYz2lNLZg+FD4of/sYvvLJmC9vfaOOfsJAwNf0CHZHLLu9taWogFC6tJMoBd0G+64a82Cg1JzPlwblfytU8zBITqJry+ffsrspqO4tR4+BQg86byyaBhT9LGr8Xij8Te38eQW5BPlqLXiSRxIfF4ek3+Z9KUo= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=Q/9kjle6; arc=none smtp.client-ip=209.85.221.52 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="Q/9kjle6" Received: by mail-wr1-f52.google.com with SMTP id ffacd0b85a97d-35f2d723ef0so876681f8f.1 for ; Tue, 11 Jun 2024 11:24:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1718130295; x=1718735095; darn=vger.kernel.org; h=cc:to:mime-version:content-transfer-encoding:fcc:subject:date:from :references:in-reply-to:message-id:from:to:cc:subject:date :message-id:reply-to; bh=2WUpad7JaJbJFbFg5yzlOkze/LhAKbg5r2VoNW+B2vY=; b=Q/9kjle6PWEIfhpbopPcd5YuNfyHQRH+9eWtLJHJk6M3b47anOJ5V8OOr5HrWD7cFg GtdBTykv7c7Wab/8FjiEsRTs8dbWLjNasibBYbypbzN3gkvhKXhnFv5X/9tVgutrabqT ifRhS6suaVFEO0cLbWdBvI4m4c5wl/XTzqvDN7kRcZUqFcJ3kjKB/hAdz0IoFqgliSne PN3GLLYakgz2XC9XynrRYvjTxFRLcHTKFBOxUWq+jKtYa+s2iEROozBVbeLg9qF8R/yh XS2eYPy7z/2wi+e7pqHSUaTTZpI32oJS6SiPI2OZWtlysgZy18hDTclVoN2Tbc8CjKDo BYTQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1718130295; x=1718735095; h=cc:to:mime-version:content-transfer-encoding:fcc:subject:date:from :references:in-reply-to:message-id:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=2WUpad7JaJbJFbFg5yzlOkze/LhAKbg5r2VoNW+B2vY=; b=lyc0v9jFH4KZt/d43I7v5ZZPSp6VNrhCepaN+pgAJEuTjwDy0xUe0HBPNw8CpsWkik PPmciwUuNBF1EWw2vOxKMQHHi/LTMw35CGvi7UwdGnDfHqTlEZtmd9pBjqzhVzIUCcod Vw1qJJmyIFW/qPZLunYzDYxegyCzFuW0rD3trvdG5Ry8Uaq3D5y9UCR0PbvuMJoRwrJG 5bwRIkR8StqO9X5aT0Ob6G1rQDIQMzeiT7XX/3J99toTwKnzQh1070JAhGtr7mFevwgm wIVdupxL0aaJ+HITw9BaY2WDu8AQOV2MLzdJ/+WYRTo2CKMGdm3TuTHxQUFzzlL39doU XQlg== X-Gm-Message-State: AOJu0YzVc9HK8xXJuPmjl7qGr44F78TjyPURR7KC2QdtU72sHx1sVMA4 g9+9jpHP5oRq994i4SC3Y6IiMjh6yzPm0ZvVLbmztOQz7fQxJCuhugdZEA== X-Google-Smtp-Source: AGHT+IFqZrNT45Oujnwtb7d1n47DFC0epvXSKTHBSUrmCgHETaObRxuvArC+HQCh/bRbeVLLoB7aWg== X-Received: by 2002:a5d:554f:0:b0:35f:1c9b:ff06 with SMTP id ffacd0b85a97d-35f1c9bffdamr5640977f8f.56.1718130295459; Tue, 11 Jun 2024 11:24:55 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-35f27600519sm4976505f8f.32.2024.06.11.11.24.55 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 11 Jun 2024 11:24:55 -0700 (PDT) Message-Id: <8d1e1eaa70b96779416f2f48a862d31a730c4521.1718130288.git.gitgitgadget@gmail.com> In-Reply-To: References: Date: Tue, 11 Jun 2024 18:24:39 +0000 Subject: [PATCH 07/16] mktree: use read_index_info to read stdin lines Fcc: Sent Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 To: git@vger.kernel.org Cc: Victoria Dye , Victoria Dye From: Victoria Dye From: Victoria Dye Replace the custom input parsing of 'mktree' with 'read_index_info()', which handles not only the 'ls-tree' output format it already handles but also the other formats compatible with 'update-index'. This lends some consistency across the commands (avoiding the need for two similar implementations for input parsing) and adds flexibility to mktree. Update 'Documentation/git-mktree.txt' to reflect the more permissive input format. Signed-off-by: Victoria Dye --- Documentation/git-mktree.txt | 17 +++-- builtin/mktree.c | 139 ++++++++++++----------------------- t/t1010-mktree.sh | 66 +++++++++++++++++ 3 files changed, 125 insertions(+), 97 deletions(-) diff --git a/Documentation/git-mktree.txt b/Documentation/git-mktree.txt index 383f09dd333..507682ed23e 100644 --- a/Documentation/git-mktree.txt +++ b/Documentation/git-mktree.txt @@ -3,7 +3,7 @@ git-mktree(1) NAME ---- -git-mktree - Build a tree-object from ls-tree formatted text +git-mktree - Build a tree-object from formatted tree entries SYNOPSIS @@ -13,15 +13,13 @@ SYNOPSIS DESCRIPTION ----------- -Reads standard input in non-recursive `ls-tree` output format, and creates -a tree object. The order of the tree entries is normalized by mktree so -pre-sorting the input is not required. The object name of the tree object -built is written to the standard output. +Reads entry information from stdin and creates a tree object from those entries. +The object name of the tree object built is written to the standard output. OPTIONS ------- -z:: - Read the NUL-terminated `ls-tree -z` output instead. + Input lines are separated with NUL rather than LF. --missing:: Allow missing objects. The default behaviour (without this option) @@ -35,6 +33,13 @@ OPTIONS optional. Note - if the `-z` option is used, lines are terminated with NUL. +INPUT FORMAT +------------ +Tree entries may be specified in any of the formats compatible with the +`--index-info` option to linkgit:git-update-index[1]. The order of the tree +entries is normalized by `mktree` so pre-sorting the input by path is not +required. + GIT --- Part of the linkgit:git[1] suite diff --git a/builtin/mktree.c b/builtin/mktree.c index 15bd908702a..5530257252d 100644 --- a/builtin/mktree.c +++ b/builtin/mktree.c @@ -6,6 +6,7 @@ #include "builtin.h" #include "gettext.h" #include "hex.h" +#include "index-info.h" #include "quote.h" #include "strbuf.h" #include "tree.h" @@ -93,123 +94,80 @@ static const char *mktree_usage[] = { NULL }; -static void mktree_line(char *buf, int nul_term_line, int allow_missing, - struct tree_entry_array *arr) +struct mktree_line_data { + struct tree_entry_array *arr; + int allow_missing; +}; + +static int mktree_line(unsigned int mode, struct object_id *oid, + enum object_type obj_type, int stage UNUSED, + const char *path, void *cbdata) { - char *ptr, *ntr; - const char *p; - unsigned mode; - enum object_type mode_type; /* object type derived from mode */ - enum object_type obj_type; /* object type derived from sha */ + struct mktree_line_data *data = cbdata; + enum object_type mode_type = object_type(mode); struct object_info oi = OBJECT_INFO_INIT; - char *path, *to_free = NULL; - struct object_id oid; + enum object_type parsed_obj_type; - ptr = buf; - /* - * Read non-recursive ls-tree output format: - * mode SP type SP sha1 TAB name - */ - mode = strtoul(ptr, &ntr, 8); - if (ptr == ntr || !ntr || *ntr != ' ') - die("input format error: %s", buf); - ptr = ntr + 1; /* type */ - ntr = strchr(ptr, ' '); - if (!ntr || parse_oid_hex(ntr + 1, &oid, &p) || - *p != '\t') - die("input format error: %s", buf); - - /* It is perfectly normal if we do not have a commit from a submodule */ - if (S_ISGITLINK(mode)) - allow_missing = 1; - - - *ntr++ = 0; /* now at the beginning of SHA1 */ - - path = (char *)p + 1; /* at the beginning of name */ - if (!nul_term_line && path[0] == '"') { - struct strbuf p_uq = STRBUF_INIT; - if (unquote_c_style(&p_uq, path, NULL)) - die("invalid quoting"); - path = to_free = strbuf_detach(&p_uq, NULL); - } + if (obj_type && mode_type != obj_type) + die("object type (%s) doesn't match mode type (%s)", + type_name(obj_type), type_name(mode_type)); - /* - * Object type is redundantly derivable three ways. - * These should all agree. - */ - mode_type = object_type(mode); - if (mode_type != type_from_string(ptr)) { - die("entry '%s' object type (%s) doesn't match mode type (%s)", - path, ptr, type_name(mode_type)); - } + oi.typep = &parsed_obj_type; - /* Check the type of object identified by oid without fetching objects */ - oi.typep = &obj_type; - if (oid_object_info_extended(the_repository, &oid, &oi, + if (oid_object_info_extended(the_repository, oid, &oi, OBJECT_INFO_LOOKUP_REPLACE | OBJECT_INFO_QUICK | OBJECT_INFO_SKIP_FETCH_OBJECT) < 0) - obj_type = -1; + parsed_obj_type = -1; - if (obj_type < 0) { - if (allow_missing) { - ; /* no problem - missing objects are presumed to be of the right type */ + if (parsed_obj_type < 0) { + if (data->allow_missing || S_ISGITLINK(mode)) { + ; /* no problem - missing objects & submodules are presumed to be of the right type */ } else { - die("entry '%s' object %s is unavailable", path, oid_to_hex(&oid)); - } - } else { - if (obj_type != mode_type) { - /* - * The object exists but is of the wrong type. - * This is a problem regardless of allow_missing - * because the new tree entry will never be correct. - */ - die("entry '%s' object %s is a %s but specified type was (%s)", - path, oid_to_hex(&oid), type_name(obj_type), type_name(mode_type)); + die("entry '%s' object %s is unavailable", path, oid_to_hex(oid)); } + } else if (parsed_obj_type != mode_type) { + /* + * The object exists but is of the wrong type. + * This is a problem regardless of allow_missing + * because the new tree entry will never be correct. + */ + die("entry '%s' object %s is a %s but specified type was (%s)", + path, oid_to_hex(oid), type_name(parsed_obj_type), type_name(mode_type)); } - append_to_tree(mode, &oid, path, arr); - free(to_free); + append_to_tree(mode, oid, path, data->arr); + return 0; } int cmd_mktree(int ac, const char **av, const char *prefix) { - struct strbuf sb = STRBUF_INIT; struct object_id oid; int nul_term_line = 0; - int allow_missing = 0; int is_batch_mode = 0; - int got_eof = 0; struct tree_entry_array arr = { 0 }; - strbuf_getline_fn getline_fn; + struct mktree_line_data mktree_line_data = { .arr = &arr }; + int ret; const struct option option[] = { OPT_BOOL('z', NULL, &nul_term_line, N_("input is NUL terminated")), - OPT_BOOL(0, "missing", &allow_missing, N_("allow missing objects")), + OPT_BOOL(0, "missing", &mktree_line_data.allow_missing, N_("allow missing objects")), OPT_BOOL(0, "batch", &is_batch_mode, N_("allow creation of more than one tree")), OPT_END() }; ac = parse_options(ac, av, prefix, option, mktree_usage, 0); - getline_fn = nul_term_line ? strbuf_getline_nul : strbuf_getline_lf; - - while (!got_eof) { - while (1) { - if (getline_fn(&sb, stdin) == EOF) { - got_eof = 1; - break; - } - if (sb.buf[0] == '\0') { - /* empty lines denote tree boundaries in batch mode */ - if (is_batch_mode) - break; - die("input format error: (blank line only valid in batch mode)"); - } - mktree_line(sb.buf, nul_term_line, allow_missing, &arr); - } - if (is_batch_mode && got_eof && arr.nr < 1) { + + do { + ret = read_index_info(nul_term_line, mktree_line, &mktree_line_data); + if (ret < 0) + break; + + /* empty lines denote tree boundaries in batch mode */ + if (ret > 0 && !is_batch_mode) + die("input format error: (blank line only valid in batch mode)"); + + if (is_batch_mode && !ret && arr.nr < 1) { /* * Execution gets here if the last tree entry is terminated with a * new-line. The final new-line has been made optional to be @@ -222,9 +180,8 @@ int cmd_mktree(int ac, const char **av, const char *prefix) fflush(stdout); } clear_tree_entry_array(&arr); /* reset tree entry buffer for re-use in batch mode */ - } + } while (ret > 0); release_tree_entry_array(&arr); - strbuf_release(&sb); - return 0; + return !!ret; } diff --git a/t/t1010-mktree.sh b/t/t1010-mktree.sh index 22875ba598c..9b2ab0c97ad 100755 --- a/t/t1010-mktree.sh +++ b/t/t1010-mktree.sh @@ -54,11 +54,36 @@ test_expect_success 'ls-tree output in wrong order given to mktree (2)' ' test_cmp tree.withsub actual ' +test_expect_success '--batch creates multiple trees' ' + cat top >multi-tree && + echo "" >>multi-tree && + cat top.withsub >>multi-tree && + + cat tree >expect && + cat tree.withsub >>expect && + git mktree --batch actual && + test_cmp expect actual +' + test_expect_success 'allow missing object with --missing' ' git mktree --missing actual && test_cmp tree.missing actual ' +test_expect_success 'mktree with invalid submodule OIDs' ' + # non-existent OID - ok + printf "160000 commit $(test_oid numeric)\tA\n" >in && + git mktree tree.actual && + git ls-tree $(cat tree.actual) >actual && + test_cmp in actual && + + # existing OID, wrong type - error + tree_oid="$(cat tree)" && + printf "160000 commit $tree_oid\tA" | + test_must_fail git mktree 2>err && + grep "object $tree_oid is a tree but specified type was (commit)" err +' + test_expect_success 'mktree refuses to read ls-tree -r output (1)' ' test_must_fail git mktree err && + grep "blank line only valid in batch mode" err && + + # bad whitespace + printf "100644 blob $EMPTY_BLOB A" | + test_must_fail git mktree 2>err && + grep "malformed input line" err && + + # invalid type + printf "100644 bad $EMPTY_BLOB\tA" | + test_must_fail git mktree 2>err && + grep "invalid object type" err && + + # invalid OID length + printf "100755 blob abc123\tA" | + test_must_fail git mktree 2>err && + grep "malformed input line" err && + + # bad quoting + printf "100644 blob $EMPTY_BLOB\t\"A" | + test_must_fail git mktree 2>err && + grep "bad quoting of path name" err +' + +test_expect_success 'mktree fails on mode mismatch' ' + tree_oid="$(cat tree)" && + + # mode-type mismatch + printf "100644 tree $tree_oid\tA" | + test_must_fail git mktree 2>err && + grep "object type (tree) doesn${SQ}t match mode type (blob)" err && + + # mode-object mismatch (no --missing) + printf "100644 $tree_oid\tA" | + test_must_fail git mktree 2>err && + grep "object $tree_oid is a tree but specified type was (blob)" err +' + test_done From patchwork Tue Jun 11 18:24:40 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Victoria Dye X-Patchwork-Id: 13694109 Received: from mail-lf1-f43.google.com (mail-lf1-f43.google.com [209.85.167.43]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D7F7278C70 for ; Tue, 11 Jun 2024 18:24:58 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.167.43 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1718130300; cv=none; b=RINVnutNlgaTMP2kcjIptGOY2A6HYbLeJ9E9VORq2M5uOXL+T8Mzcr9tnmH4W0Zwei61TEPMCdqukg6Rr1pX6oNQ9jcWOpVtbwTfBS+r6xB1I8uyu8uSoYOf5jJtO1DmG/TvOAAwEKrQIesuhLbpyeRt88sd7rOHTW5Z6+ukLlk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1718130300; c=relaxed/simple; bh=fo+KRucjZW9OQPdxvhlJmXOueEPQO90gusIsSSwSt1o=; h=Message-Id:In-Reply-To:References:From:Date:Subject:Content-Type: MIME-Version:To:Cc; b=eSVj22hM0jOrTbHq/Jsge2e0oR6+Happq+5Mz0iiTKleWIyi9zbXCUOpZQCsaSWXopRNhIUhQgWXaZKjGHqgFH6hy3o9lVRhaTP9Izdxv3bBmNOGDacn7LmFwfFlqWh9AsSyqLtXaz5ilm4VRnkddy673k34Ipq0w7Ai37/T74M= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=dqD/3Z51; arc=none smtp.client-ip=209.85.167.43 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="dqD/3Z51" Received: by mail-lf1-f43.google.com with SMTP id 2adb3069b0e04-52c89d6b4adso2714907e87.3 for ; Tue, 11 Jun 2024 11:24:58 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1718130297; x=1718735097; darn=vger.kernel.org; h=cc:to:mime-version:content-transfer-encoding:fcc:subject:date:from :references:in-reply-to:message-id:from:to:cc:subject:date :message-id:reply-to; bh=DEB9O27yrASduCiokIYNf0odHRRtLKAjE4kaiaGLMg4=; b=dqD/3Z51EkbNffMJ/picpSPfy0XITfCd9HrtsQTtxB6wAtRZ13PkL65SFxEPvkt+QO ajLjGCuQrVR1BKRHyTtaCdRJIZ5G21VWSRxZLfDrtPhAMnMadFkVGhJPIIWlQzjQQFt4 kxcrzM3neA0R8fpvF8VK9lIUiwtCPQyfhCK0/z3jGX+IciPV069Qnboa6JK7r9VpY9rg l7afBeQlFKebMbDEwYc2PPhhpzdZwJYZ7lRWZU+Q42mv5Z57DKSX/ssu4vsJ/JyOXkBk tnfbdlvVy8f2mJiyHkcVm1dWllrYXMKkgRfaWCvp5QEV0GYVoi09s2oFNNngwPt2QrDj FRiw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1718130297; x=1718735097; h=cc:to:mime-version:content-transfer-encoding:fcc:subject:date:from :references:in-reply-to:message-id:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=DEB9O27yrASduCiokIYNf0odHRRtLKAjE4kaiaGLMg4=; b=cio8fgLvPkJatxQNiiePsufYwz/O0JqvhfA0XyAN8gh4t88FVeuobXItJ+9eMlvKiK roktzZmEe0Z+Ebc59V+j2JisO0dRMPmdhsTpjtSsHWsnuM9OvUzAfwhiNnypdVzaifGw au3csd99SU6gsU+7jcTYTVQwDT9PN8NIfQncIDbgRw6SBCO7BMAOUAItOGl1kjUzeh8o D+5TU+BnkWECfVgD0FdZoiyz/h5KilLFGPy1UXVgihCFBpweUdJAX8O3LsNTVT/aLmX9 f/Jn3HkWh8Y3CckrV18BClCBWglSsu/IJmujqPU9YKaS+W5IFfslsMRH3woC7rLtPbEx yOsQ== X-Gm-Message-State: AOJu0Yzsj/cSza+U67/PgZ9ZNSqVKiimOKHdwemcuxvwI9TBxV0jHVOP zHWB5/fQ84mhFrAlS5zN0Qoa5QyMUQ2GuHTdS7ziUAc+Uxks5tg6qc4Z0A== X-Google-Smtp-Source: AGHT+IHqh6L06EpA6BgcxM6nXwgjh6tqiPrM9TLDehfNZEsSfCEHRbQoqdM6GzZjgt28aXdh8jPG6Q== X-Received: by 2002:a19:5e10:0:b0:52c:99c9:bef6 with SMTP id 2adb3069b0e04-52c99c9c02dmr35555e87.7.1718130296713; Tue, 11 Jun 2024 11:24:56 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-4225443988dsm32739775e9.47.2024.06.11.11.24.55 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 11 Jun 2024 11:24:55 -0700 (PDT) Message-Id: In-Reply-To: References: Date: Tue, 11 Jun 2024 18:24:40 +0000 Subject: [PATCH 08/16] mktree: add a --literally option Fcc: Sent Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 To: git@vger.kernel.org Cc: Victoria Dye , Victoria Dye From: Victoria Dye From: Victoria Dye Add the '--literally' option to 'git mktree' to allow constructing a tree with invalid contents. For now, the only change this represents compared to the normal 'git mktree' behavior is no longer sorting the inputs; in later commits, deduplicaton and path validation will be added to the command and '--literally' will skip those as well. Certain tests use 'git mktree' to intentionally generate corrupt trees. Update these tests to use '--literally' so that they continue functioning properly when additional input cleanup & validation is added to the base command. Note that, because 'mktree --literally' does not sort entries, some of the tests are updated to provide their inputs in tree order; otherwise, the test would fail with an "incorrect order" error instead of the error the test expects. Signed-off-by: Victoria Dye --- Documentation/git-mktree.txt | 9 ++++++- builtin/mktree.c | 36 +++++++++++++++++++++++---- t/t1010-mktree.sh | 40 ++++++++++++++++++++++++++++++ t/t1014-read-tree-confusing.sh | 6 ++--- t/t1450-fsck.sh | 4 +-- t/t1601-index-bogus.sh | 2 +- t/t1700-split-index.sh | 6 ++--- t/t7008-filter-branch-null-sha1.sh | 6 ++--- t/t7417-submodule-path-url.sh | 2 +- t/t7450-bad-git-dotfiles.sh | 8 +++--- 10 files changed, 96 insertions(+), 23 deletions(-) diff --git a/Documentation/git-mktree.txt b/Documentation/git-mktree.txt index 507682ed23e..fb07e40cef0 100644 --- a/Documentation/git-mktree.txt +++ b/Documentation/git-mktree.txt @@ -9,7 +9,7 @@ git-mktree - Build a tree-object from formatted tree entries SYNOPSIS -------- [verse] -'git mktree' [-z] [--missing] [--batch] +'git mktree' [-z] [--missing] [--literally] [--batch] DESCRIPTION ----------- @@ -27,6 +27,13 @@ OPTIONS object. This option has no effect on the treatment of gitlink entries (aka "submodules") which are always allowed to be missing. +--literally:: + Create the tree from the tree entries provided to stdin in the order + they are provided without performing additional sorting, deduplication, + or path validation on them. This option is primarily useful for creating + invalid tree objects to use in tests of how Git deals with various forms + of tree corruption. + --batch:: Allow building of more than one tree object before exiting. Each tree is separated by a single blank line. The final newline is diff --git a/builtin/mktree.c b/builtin/mktree.c index 5530257252d..48019448c1f 100644 --- a/builtin/mktree.c +++ b/builtin/mktree.c @@ -45,11 +45,11 @@ static void release_tree_entry_array(struct tree_entry_array *arr) } static void append_to_tree(unsigned mode, struct object_id *oid, const char *path, - struct tree_entry_array *arr) + struct tree_entry_array *arr, int literally) { struct tree_entry *ent; size_t len = strlen(path); - if (strchr(path, '/')) + if (!literally && strchr(path, '/')) die("path %s contains slash", path); FLEX_ALLOC_MEM(ent, name, path, len); @@ -89,14 +89,35 @@ static void write_tree(struct tree_entry_array *arr, struct object_id *oid) strbuf_release(&buf); } +static void write_tree_literally(struct tree_entry_array *arr, + struct object_id *oid) +{ + struct strbuf buf; + size_t size = 0; + + for (size_t i = 0; i < arr->nr; i++) + size += 32 + arr->entries[i]->len; + + strbuf_init(&buf, size); + for (size_t i = 0; i < arr->nr; i++) { + struct tree_entry *ent = arr->entries[i]; + strbuf_addf(&buf, "%o %s%c", ent->mode, ent->name, '\0'); + strbuf_add(&buf, ent->oid.hash, the_hash_algo->rawsz); + } + + write_object_file(buf.buf, buf.len, OBJ_TREE, oid); + strbuf_release(&buf); +} + static const char *mktree_usage[] = { - "git mktree [-z] [--missing] [--batch]", + "git mktree [-z] [--missing] [--literally] [--batch]", NULL }; struct mktree_line_data { struct tree_entry_array *arr; int allow_missing; + int literally; }; static int mktree_line(unsigned int mode, struct object_id *oid, @@ -136,7 +157,7 @@ static int mktree_line(unsigned int mode, struct object_id *oid, path, oid_to_hex(oid), type_name(parsed_obj_type), type_name(mode_type)); } - append_to_tree(mode, oid, path, data->arr); + append_to_tree(mode, oid, path, data->arr, data->literally); return 0; } @@ -152,6 +173,8 @@ int cmd_mktree(int ac, const char **av, const char *prefix) const struct option option[] = { OPT_BOOL('z', NULL, &nul_term_line, N_("input is NUL terminated")), OPT_BOOL(0, "missing", &mktree_line_data.allow_missing, N_("allow missing objects")), + OPT_BOOL(0, "literally", &mktree_line_data.literally, + N_("do not sort, deduplicate, or validate paths of tree entries")), OPT_BOOL(0, "batch", &is_batch_mode, N_("allow creation of more than one tree")), OPT_END() }; @@ -175,7 +198,10 @@ int cmd_mktree(int ac, const char **av, const char *prefix) */ ; /* skip creating an empty tree */ } else { - write_tree(&arr, &oid); + if (mktree_line_data.literally) + write_tree_literally(&arr, &oid); + else + write_tree(&arr, &oid); puts(oid_to_hex(&oid)); fflush(stdout); } diff --git a/t/t1010-mktree.sh b/t/t1010-mktree.sh index 9b2ab0c97ad..e0687cb529f 100755 --- a/t/t1010-mktree.sh +++ b/t/t1010-mktree.sh @@ -133,4 +133,44 @@ test_expect_success 'mktree fails on mode mismatch' ' grep "object $tree_oid is a tree but specified type was (blob)" err ' +test_expect_success '--literally can create invalid trees' ' + tree_oid="$(cat tree)" && + blob_oid="$(git rev-parse ${tree_oid}:one)" && + + # duplicate entries + { + printf "040000 tree $tree_oid\tmy-tree\n" && + printf "100644 blob $blob_oid\ttest-file\n" && + printf "100755 blob $blob_oid\ttest-file\n" + } | git mktree --literally >tree.bad && + git cat-file tree $(cat tree.bad) >top.bad && + test_must_fail git hash-object --stdin -t tree err && + grep "contains duplicate file entries" err && + + # disallowed path + { + printf "100644 blob $blob_oid\t.git\n" + } | git mktree --literally >tree.bad && + git cat-file tree $(cat tree.bad) >top.bad && + test_must_fail git hash-object --stdin -t tree err && + grep "contains ${SQ}.git${SQ}" err && + + # nested entry + { + printf "100644 blob $blob_oid\tdeeper/my-file\n" + } | git mktree --literally >tree.bad && + git cat-file tree $(cat tree.bad) >top.bad && + test_must_fail git hash-object --stdin -t tree err && + grep "contains full pathnames" err && + + # bad entry ordering + { + printf "100644 blob $blob_oid\tB\n" && + printf "040000 tree $tree_oid\tA\n" + } | git mktree --literally >tree.bad && + git cat-file tree $(cat tree.bad) >top.bad && + test_must_fail git hash-object --stdin -t tree err && + grep "not properly sorted" err +' + test_done diff --git a/t/t1014-read-tree-confusing.sh b/t/t1014-read-tree-confusing.sh index 8ea8d36818b..762eb789704 100755 --- a/t/t1014-read-tree-confusing.sh +++ b/t/t1014-read-tree-confusing.sh @@ -30,13 +30,13 @@ while read path pretty; do esac test_expect_success "reject $pretty at end of path" ' printf "100644 blob %s\t%s" "$blob" "$path" >tree && - bogus=$(git mktree tree && - bogus=$(git mktree tree && - ok=$(git mktree badtree && - badtree=$(git mktree out && test_grep "$badtree" out && test_grep "error in tree .*contains duplicate file entries" out @@ -614,7 +614,7 @@ while read name path pretty; do tree=$(git rev-parse HEAD^{tree}) && value=$(eval "echo \$$type") && printf "$mode $type %s\t%s" "$value" "$path" >bad && - bad_tree=$(git mktree out && test_grep "warning.*tree $bad_tree" out )' diff --git a/t/t1601-index-bogus.sh b/t/t1601-index-bogus.sh index 4171f1e1410..54e8ae038b7 100755 --- a/t/t1601-index-bogus.sh +++ b/t/t1601-index-bogus.sh @@ -4,7 +4,7 @@ test_description='test handling of bogus index entries' . ./test-lib.sh test_expect_success 'create tree with null sha1' ' - tree=$(printf "160000 commit $ZERO_OID\\tbroken\\n" | git mktree) + tree=$(printf "160000 commit $ZERO_OID\\tbroken\\n" | git mktree --literally) ' test_expect_success 'read-tree refuses to read null sha1' ' diff --git a/t/t1700-split-index.sh b/t/t1700-split-index.sh index ac4a5b2734c..97b58aa3cca 100755 --- a/t/t1700-split-index.sh +++ b/t/t1700-split-index.sh @@ -478,12 +478,12 @@ test_expect_success 'writing split index with null sha1 does not write cache tre git config splitIndex.maxPercentChange 0 && git commit -m "commit" && { - git ls-tree HEAD && - printf "160000 commit $ZERO_OID\\tbroken\\n" + printf "160000 commit $ZERO_OID\\tbroken\\n" && + git ls-tree HEAD } >broken-tree && echo "add broken entry" >msg && - tree=$(git mktree broken-tree && echo "add broken entry" >msg && - tree=$(git mktree tree && sed "s/sub/sub /" tree.new && - tree=$(git -C super mktree bad-tree ) && - tree=$(git -C $dir mktree <$dir/bad-tree) + tree=$(git -C $dir mktree --literally <$dir/bad-tree) ' test_expect_success "fsck detects symlinked $name ($type)" ' @@ -261,7 +261,7 @@ test_expect_success 'fsck detects non-blob .gitmodules' ' cp ../.gitmodules subdir/file && git add subdir/file && git commit -m ok && - git ls-tree HEAD | sed s/subdir/.gitmodules/ | git mktree && + git ls-tree HEAD | sed s/subdir/.gitmodules/ | git mktree --literally && test_must_fail git fsck 2>output && test_grep gitmodulesBlob output From patchwork Tue Jun 11 18:24:41 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Victoria Dye X-Patchwork-Id: 13694111 Received: from mail-lf1-f51.google.com (mail-lf1-f51.google.com [209.85.167.51]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 26ABB7CF1F for ; Tue, 11 Jun 2024 18:25:00 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.167.51 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1718130301; cv=none; b=NGt4Rpiw13tpQDzZ+k1iugXgXHKrevK/+RD3JI/M6p9y7FjDmXlecOoNIbSU/fYVbuuOFfMekN36cj4gQ+6SQckzjlGZ80PomcqmmY6ZFeDt/HuMJGhNDZkSGwDntdBCXwQ1VMj/oRmviLVXubkMPIwcuVyA2SF/Eh60tOydsYg= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1718130301; c=relaxed/simple; bh=tGtsJofvSvhZ0ZVew0HDFJNUUpQMeJusNyNhaUODxHU=; h=Message-Id:In-Reply-To:References:From:Date:Subject:Content-Type: MIME-Version:To:Cc; b=tIhL4oVml5Aaez9aUv/IGLuzDcqHCwRs+s2fO2SrM6zjo+DVAZkkMMBSe5OGs+3uUeG8/RNN/Xt/vsdMbVO+ZHfT027KF8V4P8m/7vbF1MDBhsYpRW5sGhlafWV+60WG/Di6iPadg1/WNa5jQS+42TvYO17x758K4+k36kd4shY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=gt74/t5/; arc=none smtp.client-ip=209.85.167.51 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="gt74/t5/" Received: by mail-lf1-f51.google.com with SMTP id 2adb3069b0e04-52bc274f438so2024478e87.0 for ; Tue, 11 Jun 2024 11:24:59 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1718130298; x=1718735098; darn=vger.kernel.org; h=cc:to:mime-version:content-transfer-encoding:fcc:subject:date:from :references:in-reply-to:message-id:from:to:cc:subject:date :message-id:reply-to; bh=idJZ59EWsSrU43wYm3Pcom/btKl7U/TKe7C44QSShX4=; b=gt74/t5/QzfZuSTbPWh/+DPxSi5/mLbyGrqkCNCdXqRvtFziGf8c/fk+KT0BEaddRf T/NmRFYWckUS05BxwcHdMA4oEtt7gn/BqXOkPAOzqfOABrrsidOvcoUzUT6ZeAXRfwtX JbFdU1A3mGJPaYkpgHtXEhsjvbmq+0woQdsxkCfkjGxSxozjmbTkdyRMSy71EPjVZ9z8 jHj9R60S/dmfuPqOzP45pAN+mzG9oVJ4PUUz8TQ735lcpsBBG6tP7f3fGvm9PdjL/1pa cTgHP7b8P5IyaYgIQB4YX4ZJ+rgV7CtFJa4VAW5lVMonDIIGPJkBhS+VU/HGi92ptyUh c7aQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1718130298; x=1718735098; h=cc:to:mime-version:content-transfer-encoding:fcc:subject:date:from :references:in-reply-to:message-id:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=idJZ59EWsSrU43wYm3Pcom/btKl7U/TKe7C44QSShX4=; b=VUzb+gk5osh7QSqtKuxhYDos/Gs/7GuM3NookihWHEaIj1k5jXuKdPRxJSmgKNfD2a TNIAXLZdLWA77iBlPWefoziHT3Xb0OD8t2o3LoN1p/azkxHeoJ5SDqFKWPGTuQMWXeJ0 RiSF0ofmVGN0JUrCSmU/DhrntsBCPih51YDiHV9PkyTxahsfC6to4zfr7huLWlTrENDN OcyXZF8nP/5MxGkg/v+N4z0+63AjRT81ppAzh4h9OiohkWJ3mkkSR36RCjRYEReWdPSt Q2xrBhXVtg0aXiKqaDmhMCBewVKVN2xmWDqq1uXdep2oB8eanbk7KQhBYEIgWp9p3p4D ci8w== X-Gm-Message-State: AOJu0YyyDKDbHkR6jLtFa5SfHwy5dIon4NPr7YkqebZEEVVFPlcnXrxo GtX46EStijotF1ryqnZj/hVWGc9tjna1LrI9RZYfsE3t+1dZcdXDoMH37A== X-Google-Smtp-Source: AGHT+IFc0DgOSoee+P/sXQUaiUI8qcGB/XTjo0BYQQqPIqM9AEV4Psf5gy+T49f9qpGXrlSWGmZ2gA== X-Received: by 2002:a05:6512:3a8b:b0:52c:8a21:b2a9 with SMTP id 2adb3069b0e04-52c8a21b9e5mr5495461e87.58.1718130297857; Tue, 11 Jun 2024 11:24:57 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-42158148f66sm225962355e9.32.2024.06.11.11.24.57 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 11 Jun 2024 11:24:57 -0700 (PDT) Message-Id: <4f9f77e693cfc4fbe72a2ae739bc7e236a3b82d3.1718130288.git.gitgitgadget@gmail.com> In-Reply-To: References: Date: Tue, 11 Jun 2024 18:24:41 +0000 Subject: [PATCH 09/16] mktree: validate paths more carefully Fcc: Sent Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 To: git@vger.kernel.org Cc: Victoria Dye , Victoria Dye From: Victoria Dye From: Victoria Dye Use 'verify_path' to validate the paths provided as tree entries, ensuring we do not create entries with paths not allowed in trees (e.g., .git). Also, remove trailing slashes on directories before validating, allowing users to provide 'folder-name/' as the path for a tree object entry. Signed-off-by: Victoria Dye --- builtin/mktree.c | 20 +++++++++++++++++--- t/t1010-mktree.sh | 33 +++++++++++++++++++++++++++++++++ 2 files changed, 50 insertions(+), 3 deletions(-) diff --git a/builtin/mktree.c b/builtin/mktree.c index 48019448c1f..29e9dc6ce69 100644 --- a/builtin/mktree.c +++ b/builtin/mktree.c @@ -8,6 +8,7 @@ #include "hex.h" #include "index-info.h" #include "quote.h" +#include "read-cache-ll.h" #include "strbuf.h" #include "tree.h" #include "parse-options.h" @@ -49,10 +50,23 @@ static void append_to_tree(unsigned mode, struct object_id *oid, const char *pat { struct tree_entry *ent; size_t len = strlen(path); - if (!literally && strchr(path, '/')) - die("path %s contains slash", path); - FLEX_ALLOC_MEM(ent, name, path, len); + if (literally) { + FLEX_ALLOC_MEM(ent, name, path, len); + } else { + /* Normalize and validate entry path */ + if (S_ISDIR(mode)) { + while(len > 0 && is_dir_sep(path[len - 1])) + len--; + } + FLEX_ALLOC_MEM(ent, name, path, len); + + if (!verify_path(ent->name, mode)) + die(_("invalid path '%s'"), path); + if (strchr(ent->name, '/')) + die("path %s contains slash", path); + } + ent->mode = mode; ent->len = len; oidcpy(&ent->oid, oid); diff --git a/t/t1010-mktree.sh b/t/t1010-mktree.sh index e0687cb529f..e0263cb2bf8 100755 --- a/t/t1010-mktree.sh +++ b/t/t1010-mktree.sh @@ -173,4 +173,37 @@ test_expect_success '--literally can create invalid trees' ' grep "not properly sorted" err ' +test_expect_success 'mktree validates path' ' + tree_oid="$(cat tree)" && + blob_oid="$(git rev-parse $tree_oid:a/one)" && + head_oid="$(git rev-parse HEAD)" && + + # Valid: tree with or without trailing slash, blob without trailing slash + { + printf "040000 tree $tree_oid\tfolder1/\n" && + printf "040000 tree $tree_oid\tfolder2\n" && + printf "100644 blob $blob_oid\tfile.txt\n" + } | git mktree >actual && + + # Invalid: blob with trailing slash + printf "100644 blob $blob_oid\ttest/" | + test_must_fail git mktree 2>err && + grep "invalid path ${SQ}test/${SQ}" err && + + # Invalid: dotdot + printf "040000 tree $tree_oid\t../" | + test_must_fail git mktree 2>err && + grep "invalid path ${SQ}../${SQ}" err && + + # Invalid: dot + printf "040000 tree $tree_oid\t." | + test_must_fail git mktree 2>err && + grep "invalid path ${SQ}.${SQ}" err && + + # Invalid: .git + printf "040000 tree $tree_oid\t.git/" | + test_must_fail git mktree 2>err && + grep "invalid path ${SQ}.git/${SQ}" err +' + test_done From patchwork Tue Jun 11 18:24:42 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Victoria Dye X-Patchwork-Id: 13694112 Received: from mail-wr1-f47.google.com (mail-wr1-f47.google.com [209.85.221.47]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 62EB87CF34 for ; Tue, 11 Jun 2024 18:25:00 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.221.47 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1718130302; cv=none; b=GXhvAeWJUE9WviFKYY6DoU4A0nLWxLLINQZrNjaw6hal3USCxFIvx2sAKnMDGM7edf9FbHn/A+l2OVmywtwJ2mOQngib2OeDUKVErEx75PvB+Ph2ovwlyJIZVxwWg1Zg4lSNP+ASwrLCSn8jtEKmv6VhvJo/xX/hHw2kBcGeS+c= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1718130302; c=relaxed/simple; bh=IeL9Arc+h5lfuMHS8Ekfoo4iNG75KeLinD+rTjXX6t4=; h=Message-Id:In-Reply-To:References:From:Date:Subject:Content-Type: MIME-Version:To:Cc; b=s6JVT7O/kikub561ceIk0Q+D18FsDqehFJwOJ3B6SRLB7E2N73FoqSn2a43wiLSC5vV8a+Lc32OQFKsu1ey4E8Jd/u8AJBzfQpVzHxmUhhQx/k4X56sIttUJT1dpTPSgXTvdQCLyYiTAucTAIDV9ybOSpBVlUXcQklD31gdMyZg= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=B/WeuXs5; arc=none smtp.client-ip=209.85.221.47 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="B/WeuXs5" Received: by mail-wr1-f47.google.com with SMTP id ffacd0b85a97d-35f1a7386d5so2657593f8f.3 for ; Tue, 11 Jun 2024 11:25:00 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1718130298; x=1718735098; darn=vger.kernel.org; h=cc:to:mime-version:content-transfer-encoding:fcc:subject:date:from :references:in-reply-to:message-id:from:to:cc:subject:date :message-id:reply-to; bh=EdofFrISorqAH4y+6Ll+hv1QLg2FISrP8aD/P0mjB9Q=; b=B/WeuXs5B58MBmKPFlJ4QVLeNovtpEFSYlYlqWZLsOSMxXl9Fjy93N8fo0Cu7Rk1Ih RpEv1Wv9RmjBKgW4KUuNa7zLMSgPto7eyHLW+fXY33TPCVn9NIS9weJ94vzNVUePgJes GiV7xae6vSzLfRk22GZGlQ6gIBIqIHEa/EYJLSvbWTI0ld1B7S4BTN9GXcYEttpkIP0d VfWuOb55jUibJk68JA6ZZfd4rYanNuxqTjdidGEu7A9yIgqC4W40h6Y7oYyqN+3xUXOg m6atBzQxDU95gyOCLktlYzHwtEjFUuseswVggiKMquc3M/7qv/8N0V2ptpCZ9a1c1Kcl lQ/g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1718130298; x=1718735098; h=cc:to:mime-version:content-transfer-encoding:fcc:subject:date:from :references:in-reply-to:message-id:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=EdofFrISorqAH4y+6Ll+hv1QLg2FISrP8aD/P0mjB9Q=; b=cLOEdc4D5NNNLg96hOFGuMBhc/Jd6dY79lfVe15OdbRPLfMrt9cl8OlGw6Pw3zV3rV qQ9eorGcviXXztZR2bW0BMGPn7hnBhm96kGJDk5uqHpAGJtAukIKgJgYZlfos1MXCUJ6 5EnZdzwD9A0of+aYk2pIDnYVBtMPV3WjsBlUAy73Kju37RXk8CsVGZ1W65Z8f5sQybRo cl65L5NpAFYaeCFzlA7SnCb1qKGmhz+J97+dT1y5RfMFlIMPIfbSHEGpiGgWwcNd/YIT yPVLSfJlM70sMrs3zTpeKJLPr+wkmKzaB+d3kDYueHqv8REzsJqAU1BtPkodkf1ObMmV dYZg== X-Gm-Message-State: AOJu0YxCZYSEG2Sli/2d3d52ZmM9/UeXrA2/wO2S7GoXVqlto787q2x1 FU5+SuH9ooqwhtAwfkamT6iQsdybRhvWC2wjtZAhFZlvs0p3mLjiU9dB5A== X-Google-Smtp-Source: AGHT+IEZmFId6IPgfxnBHyRRrtahZ4Q3DIx27XqdxGO3uHMRInF3ICvmfFVLnpS05pPOdm+Y+GujNQ== X-Received: by 2002:adf:ef47:0:b0:35f:fb4:fc7d with SMTP id ffacd0b85a97d-35f0fb50139mr6332549f8f.31.1718130298548; Tue, 11 Jun 2024 11:24:58 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-35f0d512556sm10989465f8f.29.2024.06.11.11.24.58 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 11 Jun 2024 11:24:58 -0700 (PDT) Message-Id: In-Reply-To: References: Date: Tue, 11 Jun 2024 18:24:42 +0000 Subject: [PATCH 10/16] mktree: overwrite duplicate entries Fcc: Sent Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 To: git@vger.kernel.org Cc: Victoria Dye , Victoria Dye From: Victoria Dye From: Victoria Dye If multiple tree entries with the same name are provided as input to 'mktree', only write the last one to the tree. Entries are considered duplicates if they have identical names (*not* considering mode); if a blob and a tree with the same name are provided, only the last one will be written to the tree. A tree with duplicate entries is invalid (per 'git fsck'), so that condition should be avoided wherever possible. Signed-off-by: Victoria Dye --- Documentation/git-mktree.txt | 8 ++++--- builtin/mktree.c | 45 ++++++++++++++++++++++++++++++++---- t/t1010-mktree.sh | 36 +++++++++++++++++++++++++++-- 3 files changed, 80 insertions(+), 9 deletions(-) diff --git a/Documentation/git-mktree.txt b/Documentation/git-mktree.txt index fb07e40cef0..afbc846d077 100644 --- a/Documentation/git-mktree.txt +++ b/Documentation/git-mktree.txt @@ -43,9 +43,11 @@ OPTIONS INPUT FORMAT ------------ Tree entries may be specified in any of the formats compatible with the -`--index-info` option to linkgit:git-update-index[1]. The order of the tree -entries is normalized by `mktree` so pre-sorting the input by path is not -required. +`--index-info` option to linkgit:git-update-index[1]. + +The order of the tree entries is normalized by `mktree` so pre-sorting the input +by path is not required. Multiple entries provided with the same path are +deduplicated, with only the last one specified added to the tree. GIT --- diff --git a/builtin/mktree.c b/builtin/mktree.c index 29e9dc6ce69..e9e2134136f 100644 --- a/builtin/mktree.c +++ b/builtin/mktree.c @@ -15,6 +15,9 @@ #include "object-store-ll.h" struct tree_entry { + /* Internal */ + size_t order; + unsigned mode; struct object_id oid; int len; @@ -72,15 +75,49 @@ static void append_to_tree(unsigned mode, struct object_id *oid, const char *pat oidcpy(&ent->oid, oid); /* Append the update */ + ent->order = arr->nr; tree_entry_array_push(arr, ent); } -static int ent_compare(const void *a_, const void *b_) +static int ent_compare(const void *a_, const void *b_, void *ctx) { + int cmp; struct tree_entry *a = *(struct tree_entry **)a_; struct tree_entry *b = *(struct tree_entry **)b_; - return base_name_compare(a->name, a->len, a->mode, - b->name, b->len, b->mode); + int ignore_mode = *((int *)ctx); + + if (ignore_mode) + cmp = name_compare(a->name, a->len, b->name, b->len); + else + cmp = base_name_compare(a->name, a->len, a->mode, + b->name, b->len, b->mode); + return cmp ? cmp : b->order - a->order; +} + +static void sort_and_dedup_tree_entry_array(struct tree_entry_array *arr) +{ + size_t count = arr->nr; + struct tree_entry *prev = NULL; + + int ignore_mode = 1; + QSORT_S(arr->entries, arr->nr, ent_compare, &ignore_mode); + + arr->nr = 0; + for (size_t i = 0; i < count; i++) { + struct tree_entry *curr = arr->entries[i]; + if (prev && + !name_compare(prev->name, prev->len, + curr->name, curr->len)) { + FREE_AND_NULL(curr); + } else { + arr->entries[arr->nr++] = curr; + prev = curr; + } + } + + /* Sort again to order the entries for tree insertion */ + ignore_mode = 0; + QSORT_S(arr->entries, arr->nr, ent_compare, &ignore_mode); } static void write_tree(struct tree_entry_array *arr, struct object_id *oid) @@ -88,7 +125,7 @@ static void write_tree(struct tree_entry_array *arr, struct object_id *oid) struct strbuf buf; size_t size = 0; - QSORT(arr->entries, arr->nr, ent_compare); + sort_and_dedup_tree_entry_array(arr); for (size_t i = 0; i < arr->nr; i++) size += 32 + arr->entries[i]->len; diff --git a/t/t1010-mktree.sh b/t/t1010-mktree.sh index e0263cb2bf8..956692347f0 100755 --- a/t/t1010-mktree.sh +++ b/t/t1010-mktree.sh @@ -6,11 +6,16 @@ TEST_PASSES_SANITIZE_LEAK=true . ./test-lib.sh test_expect_success setup ' - for d in a a- a0 + for d in folder folder- folder0 do mkdir "$d" && echo "$d/one" >"$d/one" && git add "$d" || return 1 done && + for f in before folder.txt later + do + echo "$f" >"$f" && + git add "$f" || return 1 + done && echo zero >one && git update-index --add --info-only one && git write-tree --missing-ok >tree.missing && @@ -175,7 +180,7 @@ test_expect_success '--literally can create invalid trees' ' test_expect_success 'mktree validates path' ' tree_oid="$(cat tree)" && - blob_oid="$(git rev-parse $tree_oid:a/one)" && + blob_oid="$(git rev-parse $tree_oid:folder.txt)" && head_oid="$(git rev-parse HEAD)" && # Valid: tree with or without trailing slash, blob without trailing slash @@ -206,4 +211,31 @@ test_expect_success 'mktree validates path' ' grep "invalid path ${SQ}.git/${SQ}" err ' +test_expect_success 'mktree with duplicate entries' ' + tree_oid=$(cat tree) && + folder_oid=$(git rev-parse ${tree_oid}:folder) && + before_oid=$(git rev-parse ${tree_oid}:before) && + head_oid=$(git rev-parse HEAD) && + + { + printf "100755 blob $before_oid\ttest\n" && + printf "040000 tree $folder_oid\ttest-\n" && + printf "160000 commit $head_oid\ttest.txt\n" && + printf "040000 tree $folder_oid\ttest\n" && + printf "100644 blob $before_oid\ttest0\n" && + printf "160000 commit $head_oid\ttest-\n" + } >top.dup && + git mktree tree.actual && + + { + printf "160000 commit $head_oid\ttest-\n" && + printf "160000 commit $head_oid\ttest.txt\n" && + printf "040000 tree $folder_oid\ttest\n" && + printf "100644 blob $before_oid\ttest0\n" + } >expect && + git ls-tree $(cat tree.actual) >actual && + + test_cmp expect actual +' + test_done From patchwork Tue Jun 11 18:24:43 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Victoria Dye X-Patchwork-Id: 13694113 Received: from mail-wr1-f54.google.com (mail-wr1-f54.google.com [209.85.221.54]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A2AD57E57C for ; Tue, 11 Jun 2024 18:25:01 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.221.54 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1718130303; cv=none; b=BbGNpU61oKjkoVMZHKZuNB5a6S7RuVH5v6JgbVWeZI1yWqTmgCCTAtT2GF3UTnH/1ZPGi4MaRElTiucCEoyOmjj+wClBD2y5SX8/3biDtChS563FwOgnNjI1ey3oR6NHAkeYq68tK6A8uKCDykxM3eNiJ4+BKFRXOoSdK4T+KQE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1718130303; c=relaxed/simple; bh=GA/uYGCuc3dtJWd4YJAb9SnH9EG9gKAMvHkIu0762qc=; h=Message-Id:In-Reply-To:References:From:Date:Subject:Content-Type: MIME-Version:To:Cc; b=XStrgn4lusQ3kk9AHU/cq8AzOVHWEC1E/Ksg5x0H3TqJy4amGnn+dw/LC8Zn4P3LnwrpC54v48b41VXxHRFdDvK8rKi00P4kmybL0I9iA2OoGyBCD3tCO+TxLnTJzrfwDD9oHHFG+6Uk8hIC0+YnYoz/1/QpxPjlJLid2C232C4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=H8h1i88v; arc=none smtp.client-ip=209.85.221.54 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="H8h1i88v" Received: by mail-wr1-f54.google.com with SMTP id ffacd0b85a97d-35f06861ae6so3652209f8f.2 for ; Tue, 11 Jun 2024 11:25:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1718130299; x=1718735099; darn=vger.kernel.org; h=cc:to:mime-version:content-transfer-encoding:fcc:subject:date:from :references:in-reply-to:message-id:from:to:cc:subject:date :message-id:reply-to; bh=FMNUmGFaG7R8EPEWmcuiLS+3zud0y74cveu4YNpA5Bs=; b=H8h1i88vhcKMP5bnXIKR9D7fc5KZL1y/JMapxdTXkQvcrJW+IVc6zI0Edx9ItiMO+y GXnwunI2bXPS9CFaYRTCwYV0ThO3KW/sUR7GFnZ5r4zC86knglVmMWBbNf7xMl1eMppE FxG2rqBhXF2DJzH0qE86euXBL6uYhhoC6gCuKVveJ+yjGjt9X/Vv4ExUZnUxft0rw0IC qymOySHnOcvV+LWeLJfCWvCmhUQxq5fJdrJJU+461Cy3afH70JiwmqHIrqXX2DiSq1RT WV+4BDAfR0R8wU7ZOXicKQh9FNm75O9/XXyvTw7P2Gy7YaXnrSlZpqVbKx49cTKm8R7t 5KAw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1718130299; x=1718735099; h=cc:to:mime-version:content-transfer-encoding:fcc:subject:date:from :references:in-reply-to:message-id:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=FMNUmGFaG7R8EPEWmcuiLS+3zud0y74cveu4YNpA5Bs=; b=SjL+bj9ilTrUIKDycrBxi1gVyQ6/eRoriBzpfIN+SbpGV+BPUx37F5cyplvHoHh0sP yBedOLNqz1emFy5RXj1h1UpmltIfZ9U/4jewXTtDFxfAwhvsG2CjJgpMZ9tg23hF6UzW t83yuLR5WKRf7jVJUcjsX4N4j/gGx+2QzcRkD5hGdNqEDDxPfgeGjOuYYRG/CpFL7FCW 3u5DNK4cKm09njOO4Z+aFnMXNGnpJv8a0QcFRmzlm3eqADRtckIfbbVE9BV695O+Ninh AncsCBGs4HjRkOpYju/n9oth8micTEesjoiqpqGnblOCtMpSofPcsLBMLyU7GqrbWJmZ 2tqw== X-Gm-Message-State: AOJu0YxA9SwrNzgXecyHzHZ2cBDojY6DYWI3WomXscbs6wMSLJhBWyg0 yoWj21sXQxji17Tx5M2q8XDHoC2sCVvnB6kfjgu7lho6g8eNd4ZCwCgDOw== X-Google-Smtp-Source: AGHT+IFWfKO8YAlDsxt2sTKMgEFoXnU17tk198qGmRMnp40Dpsxl+VtYxmPBBNxBTIGQMFFgd039+Q== X-Received: by 2002:a05:6000:1f83:b0:35f:283e:9504 with SMTP id ffacd0b85a97d-35f283e9617mr4679397f8f.42.1718130299548; Tue, 11 Jun 2024 11:24:59 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-35f0f996ee3sm10446988f8f.71.2024.06.11.11.24.58 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 11 Jun 2024 11:24:58 -0700 (PDT) Message-Id: <130413f2404bb27a2ede4fb00041227c90587e8e.1718130288.git.gitgitgadget@gmail.com> In-Reply-To: References: Date: Tue, 11 Jun 2024 18:24:43 +0000 Subject: [PATCH 11/16] mktree: create tree using an in-core index Fcc: Sent Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 To: git@vger.kernel.org Cc: Victoria Dye , Victoria Dye From: Victoria Dye From: Victoria Dye Rather than manually write out the contents of a tree object file, construct an in-memory sparse index from the provided tree entries and create the tree by writing out its corresponding cache tree. This patch does not change the behavior of the 'mktree' command. However, constructing the tree this way will substantially simplify future extensions to the command's functionality, including handling deeper-than-toplevel tree entries and applying the provided entries to an existing tree. Signed-off-by: Victoria Dye --- builtin/mktree.c | 74 +++++++++++++++++++++++++++++++++++------------- 1 file changed, 55 insertions(+), 19 deletions(-) diff --git a/builtin/mktree.c b/builtin/mktree.c index e9e2134136f..12f68187221 100644 --- a/builtin/mktree.c +++ b/builtin/mktree.c @@ -4,6 +4,7 @@ * Copyright (c) Junio C Hamano, 2006, 2009 */ #include "builtin.h" +#include "cache-tree.h" #include "gettext.h" #include "hex.h" #include "index-info.h" @@ -24,6 +25,11 @@ struct tree_entry { char name[FLEX_ARRAY]; }; +static inline size_t df_path_len(size_t pathlen, unsigned int mode) +{ + return S_ISDIR(mode) ? pathlen - 1 : pathlen; +} + struct tree_entry_array { size_t nr, alloc; struct tree_entry **entries; @@ -57,17 +63,25 @@ static void append_to_tree(unsigned mode, struct object_id *oid, const char *pat if (literally) { FLEX_ALLOC_MEM(ent, name, path, len); } else { + size_t len_to_copy = len; + /* Normalize and validate entry path */ if (S_ISDIR(mode)) { - while(len > 0 && is_dir_sep(path[len - 1])) - len--; + while(len_to_copy > 0 && is_dir_sep(path[len_to_copy - 1])) + len_to_copy--; + len = len_to_copy + 1; /* add space for trailing slash */ } - FLEX_ALLOC_MEM(ent, name, path, len); + ent = xcalloc(1, st_add3(sizeof(struct tree_entry), len, 1)); + memcpy(ent->name, path, len_to_copy); if (!verify_path(ent->name, mode)) die(_("invalid path '%s'"), path); if (strchr(ent->name, '/')) die("path %s contains slash", path); + + /* Add trailing slash to dir */ + if (S_ISDIR(mode)) + ent->name[len - 1] = '/'; } ent->mode = mode; @@ -86,11 +100,14 @@ static int ent_compare(const void *a_, const void *b_, void *ctx) struct tree_entry *b = *(struct tree_entry **)b_; int ignore_mode = *((int *)ctx); - if (ignore_mode) - cmp = name_compare(a->name, a->len, b->name, b->len); - else - cmp = base_name_compare(a->name, a->len, a->mode, - b->name, b->len, b->mode); + size_t a_len = a->len, b_len = b->len; + + if (ignore_mode) { + a_len = df_path_len(a_len, a->mode); + b_len = df_path_len(b_len, b->mode); + } + + cmp = name_compare(a->name, a_len, b->name, b_len); return cmp ? cmp : b->order - a->order; } @@ -106,8 +123,8 @@ static void sort_and_dedup_tree_entry_array(struct tree_entry_array *arr) for (size_t i = 0; i < count; i++) { struct tree_entry *curr = arr->entries[i]; if (prev && - !name_compare(prev->name, prev->len, - curr->name, curr->len)) { + !name_compare(prev->name, df_path_len(prev->len, prev->mode), + curr->name, df_path_len(curr->len, curr->mode))) { FREE_AND_NULL(curr); } else { arr->entries[arr->nr++] = curr; @@ -120,24 +137,43 @@ static void sort_and_dedup_tree_entry_array(struct tree_entry_array *arr) QSORT_S(arr->entries, arr->nr, ent_compare, &ignore_mode); } +static int add_tree_entry_to_index(struct index_state *istate, + struct tree_entry *ent) +{ + struct cache_entry *ce; + struct strbuf ce_name = STRBUF_INIT; + strbuf_add(&ce_name, ent->name, ent->len); + + ce = make_cache_entry(istate, ent->mode, &ent->oid, ent->name, 0, 0); + if (!ce) + return error(_("make_cache_entry failed for path '%s'"), ent->name); + + add_index_entry(istate, ce, ADD_CACHE_JUST_APPEND); + strbuf_release(&ce_name); + return 0; +} + static void write_tree(struct tree_entry_array *arr, struct object_id *oid) { - struct strbuf buf; - size_t size = 0; + struct index_state istate = INDEX_STATE_INIT(the_repository); + istate.sparse_index = 1; sort_and_dedup_tree_entry_array(arr); - for (size_t i = 0; i < arr->nr; i++) - size += 32 + arr->entries[i]->len; - strbuf_init(&buf, size); + /* Construct an in-memory index from the provided entries */ for (size_t i = 0; i < arr->nr; i++) { struct tree_entry *ent = arr->entries[i]; - strbuf_addf(&buf, "%o %s%c", ent->mode, ent->name, '\0'); - strbuf_add(&buf, ent->oid.hash, the_hash_algo->rawsz); + + if (add_tree_entry_to_index(&istate, ent)) + die(_("failed to add tree entry '%s'"), ent->name); } - write_object_file(buf.buf, buf.len, OBJ_TREE, oid); - strbuf_release(&buf); + /* Write out new tree */ + if (cache_tree_update(&istate, WRITE_TREE_SILENT | WRITE_TREE_MISSING_OK)) + die(_("failed to write tree")); + oidcpy(oid, &istate.cache_tree->oid); + + release_index(&istate); } static void write_tree_literally(struct tree_entry_array *arr, From patchwork Tue Jun 11 18:24:44 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Victoria Dye X-Patchwork-Id: 13694114 Received: from mail-wr1-f49.google.com (mail-wr1-f49.google.com [209.85.221.49]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 36E078003A for ; Tue, 11 Jun 2024 18:25:02 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.221.49 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1718130303; cv=none; b=LfYTCMubbSExF5i8SZSuca3wcVVBtaPPYGAfNY0RfTkvLqoVNiR59wtjQG18aui54ejqCnFQOw0bMjzi5sT+6OnbSKqEIzKQl2ueyXwadIQEqA7qYvTCOaqevKYG33cRe1OqIXcAvlFH5f5wzyuShrfIg98nnzA9mD/yKE83uak= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1718130303; c=relaxed/simple; bh=vggQz5xtKp4oYLuEIP0VITAk9w6tsTow1AjNYRGSm+k=; h=Message-Id:In-Reply-To:References:From:Date:Subject:Content-Type: MIME-Version:To:Cc; b=VhmESR8TBCnxzw7ZtMZaT6pmPHZYApjRn/dALfitXBCkXWxWnhuuTicaIuwdR+Q9PdauDBxzqHbMyS7IT2c1XnF+KBusgZnXb1CwhQoQNGB/fTQaJeT28aBi9jNiY+R+LZAzLopjRcMacvIISqh2yhy2JH1m10axg48DTjWDqco= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=fJJcDw96; arc=none smtp.client-ip=209.85.221.49 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="fJJcDw96" Received: by mail-wr1-f49.google.com with SMTP id ffacd0b85a97d-35dc9cef36dso1348508f8f.3 for ; Tue, 11 Jun 2024 11:25:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1718130300; x=1718735100; darn=vger.kernel.org; h=cc:to:mime-version:content-transfer-encoding:fcc:subject:date:from :references:in-reply-to:message-id:from:to:cc:subject:date :message-id:reply-to; bh=T+pumRK8woWcFCtpeQ0wPDzTGFwi1qaB2U9Y2Hxalmg=; b=fJJcDw96h0qO3j6puRVKCRr6CxYBzjAy8EJzKmM0z8FelYylzC9+nuTMPeIi4XnBHh wi98woyIOP0RDPHqy/YKn94Jyx9EcjcungHRBem7z2bg/j8W4ntgiKWOwv6VFaeVDv8L XC0o1npowBV4TujVfscl3deBy/i7M3nlb417sj2iOeGDeZOOsYoEMyPdeplqMjQLb11Y CGdALTQkxd3uy1yJbCIqAm9gKtn8D7TE1dZzcZUQ5UtC3KPrA7VzC3lfE9roq3e23210 KV+59BFPu/3Z50SD6oq84uPwo77WAe3co8qTKxYOGscPlgm6WN8yb9dR9szKP06wk0kW /8zA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1718130300; x=1718735100; h=cc:to:mime-version:content-transfer-encoding:fcc:subject:date:from :references:in-reply-to:message-id:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=T+pumRK8woWcFCtpeQ0wPDzTGFwi1qaB2U9Y2Hxalmg=; b=WjwwcNq9sFOUfzq/eoAq8bHceia+M2j5VFckG5CdA5ldtfyYBfxqt0U0dNRZl0bgXj Of+hot7iWOXdtXZKtCz0y+1KUNiC7CAEozKLZjCneZBeyHwjbOYep8fy/uKRewDt7sdu ZKdUdfI/Sz00vMB48lP1H39GQcCowcaVutfgf1c8ZOIVtHKj2yZ/ElHEgygqcNb/t0Fc xYYfPwy7YnDwziqNBAJdezXVN6uc4buKUuw0H6WXvCU64eU9QIxseyV1x/CA+jZJorub z2ILE9hNlZ5mKp9iKfoXZ4s+bShgEpWdC4VRLY51Y9NTI8Z5pFSZriim/PDZ1Qd9qLpe B6Rw== X-Gm-Message-State: AOJu0YyxwXCmkhTRuqkQWjtjCZGDWSSVcV4aZquuHm2iCYGNIcafAL0L HOqce8huabb4R1zLJ+3cJhqPIrFLmf/SntHtAbUkxy8fkKf8EkWryD0hrw== X-Google-Smtp-Source: AGHT+IGDE4QCdiN1xX3doqqKEa2SDI3axnkx2FOjUWoiedT3a/iyoZ5lnYPHEZYDuu2V8K/SemcJSA== X-Received: by 2002:a05:6000:1541:b0:35f:24dc:ad97 with SMTP id ffacd0b85a97d-35f24dcaedcmr5277825f8f.34.1718130300197; Tue, 11 Jun 2024 11:25:00 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-35f254fb3c2sm5836625f8f.49.2024.06.11.11.24.59 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 11 Jun 2024 11:24:59 -0700 (PDT) Message-Id: <94d6615d634c4f78c88d3e01abbb27f13f85828c.1718130288.git.gitgitgadget@gmail.com> In-Reply-To: References: Date: Tue, 11 Jun 2024 18:24:44 +0000 Subject: [PATCH 12/16] mktree: use iterator struct to add tree entries to index Fcc: Sent Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 To: git@vger.kernel.org Cc: Victoria Dye , Victoria Dye From: Victoria Dye From: Victoria Dye Create 'struct tree_entry_iterator' to manage iteration through a 'struct tree_entry_array'. Using an iterator allows for conditional iteration; this functionality will be necessary in later commits when performing parallel iteration through multiple sets of tree entries. Signed-off-by: Victoria Dye --- builtin/mktree.c | 40 +++++++++++++++++++++++++++++++++++++--- 1 file changed, 37 insertions(+), 3 deletions(-) diff --git a/builtin/mktree.c b/builtin/mktree.c index 12f68187221..bee359e9978 100644 --- a/builtin/mktree.c +++ b/builtin/mktree.c @@ -137,6 +137,38 @@ static void sort_and_dedup_tree_entry_array(struct tree_entry_array *arr) QSORT_S(arr->entries, arr->nr, ent_compare, &ignore_mode); } +struct tree_entry_iterator { + struct tree_entry *current; + + /* private */ + struct { + struct tree_entry_array *arr; + size_t idx; + } priv; +}; + +static void init_tree_entry_iterator(struct tree_entry_iterator *iter, + struct tree_entry_array *arr) +{ + iter->priv.arr = arr; + iter->priv.idx = 0; + iter->current = 0 < arr->nr ? arr->entries[0] : NULL; +} + +/* + * Advance the tree entry iterator to the next entry in the array. If no entries + * remain, 'current' is set to NULL. Returns the previous 'current' value of the + * iterator. + */ +static struct tree_entry *advance_tree_entry_iterator(struct tree_entry_iterator *iter) +{ + struct tree_entry *prev = iter->current; + iter->current = (iter->priv.idx + 1) < iter->priv.arr->nr + ? iter->priv.arr->entries[++iter->priv.idx] + : NULL; + return prev; +} + static int add_tree_entry_to_index(struct index_state *istate, struct tree_entry *ent) { @@ -155,15 +187,17 @@ static int add_tree_entry_to_index(struct index_state *istate, static void write_tree(struct tree_entry_array *arr, struct object_id *oid) { + struct tree_entry_iterator iter = { NULL }; + struct tree_entry *ent; struct index_state istate = INDEX_STATE_INIT(the_repository); istate.sparse_index = 1; sort_and_dedup_tree_entry_array(arr); - /* Construct an in-memory index from the provided entries */ - for (size_t i = 0; i < arr->nr; i++) { - struct tree_entry *ent = arr->entries[i]; + init_tree_entry_iterator(&iter, arr); + /* Construct an in-memory index from the provided entries & base tree */ + while ((ent = advance_tree_entry_iterator(&iter))) { if (add_tree_entry_to_index(&istate, ent)) die(_("failed to add tree entry '%s'"), ent->name); } From patchwork Tue Jun 11 18:24:45 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Victoria Dye X-Patchwork-Id: 13694115 Received: from mail-wr1-f47.google.com (mail-wr1-f47.google.com [209.85.221.47]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 3506181AC8 for ; Tue, 11 Jun 2024 18:25:03 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.221.47 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1718130304; cv=none; b=rJQk9wT+o6lm/1uYnijWRFkbZCBVbKFBbm8A09ARcjJttMRfkAa5l+tU+m6HVch87zhQR2CG7NepUdBqN/2wHczlgZF/zunUdjViyF1oK0d8FWtAKRnzQ34kfn+j6vKctI+pdSOAfuP5TVAumwWdUOd6ReHlAwv0OGv4yd3ODik= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1718130304; c=relaxed/simple; bh=juJ5A/iY2RufRgHGN13pLErLvvIEMMqU9+2X4YJBL28=; h=Message-Id:In-Reply-To:References:From:Date:Subject:Content-Type: MIME-Version:To:Cc; b=JuCOmTQ9ZniLBmwM+tF0eIEdKZ0HvajvBerZYMp+BXZmAWGXVXGWLUD0XtdZHh3+2KzycFF9/5X2HqftXyQQkrLqzwzNZLzvm4iu0yA5buKDQ6ngk6RkXWQvv/u3Ka+8JLQpBMrm3f/JIpdND6R7PbVkpt5v0PZo0Obb5PapOQM= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=ckyKpG8K; arc=none smtp.client-ip=209.85.221.47 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="ckyKpG8K" Received: by mail-wr1-f47.google.com with SMTP id ffacd0b85a97d-35f1c567ae4so2719869f8f.1 for ; Tue, 11 Jun 2024 11:25:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1718130301; x=1718735101; darn=vger.kernel.org; h=cc:to:mime-version:content-transfer-encoding:fcc:subject:date:from :references:in-reply-to:message-id:from:to:cc:subject:date :message-id:reply-to; bh=LIbDgrZZzLffzDsPZ7EVcEJznmdLDB8xTVpvzkl5Em8=; b=ckyKpG8KCQJz9wv1cNBRUiasMoPGm2FHRU8U6mHPrsjAa+pzjlk7UtFb9Uexh9mtJt v8rXzqVrwrNbq05Y73U0yP2F8GM5Xh14EXb5BxjSUurQ+UAdd0EMacjn7D+tp8MCgvro ALhqmnoYG3TaFA83CwFvSjR8px5rXMFJFlH7hl8DcYmBCyEOyYAf4VyOHE7xqjVfzNjt X7/zqU0lXXh+Fp+OP1llJdMFZRM5MCjbLiRlkrR9/ALtQ5umF+EkDgTuP+joS4jgeyEL PYuQhJFx527DG/Xv5b3v8uVIs0a5Wiv4kmu3fUgJMhbohy3quaEaQyx1if+V85O3RyU4 a2Ug== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1718130301; x=1718735101; h=cc:to:mime-version:content-transfer-encoding:fcc:subject:date:from :references:in-reply-to:message-id:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=LIbDgrZZzLffzDsPZ7EVcEJznmdLDB8xTVpvzkl5Em8=; b=aHeBcLRY/So7nztZQm+u60NRq3LRNAGT+dCTUGRWi0GkbTrzvMHzEeGH13+c+YkUhk Oxu9+COJTGKKYm8ntnsi1ge56gVf5oNd94p7weelE/gDNA6g0nKaDFa0U1C6I/0HtMLd 2YAq3BReuTK9gVEZSGQHIEcggd5M/TluiTTi3V2w7rSJmSptG7LStUw/RUji1KVsk40R j8jobDWg+1bG25tQT++RqfPbVXNIbve1+Of2v/JvplNb1p7zZz1McPnSGMddYSOzrNpW A3h++phXVuUAhcr2TTOY3CL46hznwOm4c3sMnpwwT13vMdk8+V5ZBFlxr8CtLFY07CTR cvDQ== X-Gm-Message-State: AOJu0Ywialrct5PgHbE77eBX8oNvHPYlP+Ih1cK6yHpbiaAexBXv35N3 1laCQtSm3PcgHBW6jUU11emBemYYVl4kFxdMQq8ym+7CWz+ENTJky0kxGQ== X-Google-Smtp-Source: AGHT+IHGs0ag/KRifTEUft8rRrCTklG+krgc5+4mvBNtK/61u3IIz3Y3DiQuC4rlo2WEl11vaS0iqg== X-Received: by 2002:a5d:4644:0:b0:35f:1522:10b1 with SMTP id ffacd0b85a97d-35f1522121cmr6567511f8f.52.1718130301263; Tue, 11 Jun 2024 11:25:01 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-35f2169fc45sm7016616f8f.10.2024.06.11.11.25.00 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 11 Jun 2024 11:25:00 -0700 (PDT) Message-Id: <68acdd3c5ee266fd0c1d3ae45f69af4ef9012e07.1718130288.git.gitgitgadget@gmail.com> In-Reply-To: References: Date: Tue, 11 Jun 2024 18:24:45 +0000 Subject: [PATCH 13/16] mktree: add directory-file conflict hashmap Fcc: Sent Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 To: git@vger.kernel.org Cc: Victoria Dye , Victoria Dye From: Victoria Dye From: Victoria Dye Create a hashmap member of a 'struct tree_entry_array' that contains all of the (de-duplicated) provided tree entries, indexed by the hash of their path with *no* trailing slash. This hashmap will be used in a later commit to avoid adding a file to an existing tree that has the same path as a directory, or vice versa. Signed-off-by: Victoria Dye --- builtin/mktree.c | 39 +++++++++++++++++++++++++++++++++++++++ 1 file changed, 39 insertions(+) diff --git a/builtin/mktree.c b/builtin/mktree.c index bee359e9978..09b3c5c6244 100644 --- a/builtin/mktree.c +++ b/builtin/mktree.c @@ -16,6 +16,8 @@ #include "object-store-ll.h" struct tree_entry { + struct hashmap_entry ent; + /* Internal */ size_t order; @@ -33,8 +35,33 @@ static inline size_t df_path_len(size_t pathlen, unsigned int mode) struct tree_entry_array { size_t nr, alloc; struct tree_entry **entries; + + struct hashmap df_name_hash; }; +static int df_name_hash_cmp(const void *cmp_data UNUSED, + const struct hashmap_entry *eptr, + const struct hashmap_entry *entry_or_key, + const void *keydata UNUSED) +{ + const struct tree_entry *e1, *e2; + size_t e1_len, e2_len; + + e1 = container_of(eptr, const struct tree_entry, ent); + e2 = container_of(entry_or_key, const struct tree_entry, ent); + + e1_len = df_path_len(e1->len, e1->mode); + e2_len = df_path_len(e2->len, e2->mode); + + return e1_len != e2_len || + name_compare(e1->name, e1_len, e2->name, e2_len); +} + +static void init_tree_entry_array(struct tree_entry_array *arr) +{ + hashmap_init(&arr->df_name_hash, df_name_hash_cmp, NULL, 0); +} + static void tree_entry_array_push(struct tree_entry_array *arr, struct tree_entry *ent) { ALLOC_GROW(arr->entries, arr->nr + 1, arr->alloc); @@ -43,6 +70,7 @@ static void tree_entry_array_push(struct tree_entry_array *arr, struct tree_entr static void clear_tree_entry_array(struct tree_entry_array *arr) { + hashmap_clear(&arr->df_name_hash); for (size_t i = 0; i < arr->nr; i++) FREE_AND_NULL(arr->entries[i]); arr->nr = 0; @@ -50,6 +78,7 @@ static void clear_tree_entry_array(struct tree_entry_array *arr) static void release_tree_entry_array(struct tree_entry_array *arr) { + hashmap_clear(&arr->df_name_hash); FREE_AND_NULL(arr->entries); arr->nr = arr->alloc = 0; } @@ -135,6 +164,14 @@ static void sort_and_dedup_tree_entry_array(struct tree_entry_array *arr) /* Sort again to order the entries for tree insertion */ ignore_mode = 0; QSORT_S(arr->entries, arr->nr, ent_compare, &ignore_mode); + + /* Finally, initialize the directory-file conflict hash map */ + for (size_t i = 0; i < count; i++) { + struct tree_entry *curr = arr->entries[i]; + hashmap_entry_init(&curr->ent, + memhash(curr->name, df_path_len(curr->len, curr->mode))); + hashmap_put(&arr->df_name_hash, &curr->ent); + } } struct tree_entry_iterator { @@ -302,6 +339,8 @@ int cmd_mktree(int ac, const char **av, const char *prefix) ac = parse_options(ac, av, prefix, option, mktree_usage, 0); + init_tree_entry_array(&arr); + do { ret = read_index_info(nul_term_line, mktree_line, &mktree_line_data); if (ret < 0) From patchwork Tue Jun 11 18:24:46 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Victoria Dye X-Patchwork-Id: 13694116 Received: from mail-wm1-f41.google.com (mail-wm1-f41.google.com [209.85.128.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A8FB884D14 for ; Tue, 11 Jun 2024 18:25:04 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.41 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1718130306; cv=none; b=Z9ckCDNfVoh57TeB9ZgUXyXSs97xUjN3zqgxCNliEdLAxyko6XM9xBODxHBcupiTqgfDFAK2E6N+A9dfjWJUTiZW8LGz7fk0UiUifrBwZdyIVNGfEqk/D0gOQUqitShLu8KDqsNQEUjE3ye2wXY19iL+gXZKdR6kGj9Wnhj1fTY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1718130306; c=relaxed/simple; bh=EYIU2ZUTPNHcI8Ir1ovFkeSNyZ5uNmYuDke9nuYLUW8=; h=Message-Id:In-Reply-To:References:From:Date:Subject:Content-Type: MIME-Version:To:Cc; b=svZHdfhqGp56Cb6xuWGdfYXLBLYxlAMkcYy/UFXfitqeoYzE9pilm7egGcG59RDMMWjEqoG72c8WvtrpmT0g+QearKqSsY7LzjD7DqR47oVyXz7WeV+EsX8HDQ+hzvwvO98UD0aDPN4ivDeiCZAJXh/kN/AuAvxzt3BaoLhSvEY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=SEcuC0dS; arc=none smtp.client-ip=209.85.128.41 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="SEcuC0dS" Received: by mail-wm1-f41.google.com with SMTP id 5b1f17b1804b1-42281d8cd2dso327405e9.3 for ; Tue, 11 Jun 2024 11:25:04 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1718130302; x=1718735102; darn=vger.kernel.org; h=cc:to:mime-version:content-transfer-encoding:fcc:subject:date:from :references:in-reply-to:message-id:from:to:cc:subject:date :message-id:reply-to; bh=NV51om7aiHQeQkc031BzkWonrWwPeiG+D47N+moPbpw=; b=SEcuC0dSNNqRQBxqpElkiMN8EKbeXhea8JFTN8XYsSno/IjDQWzjZMTpG3zE3BnJ3N slN84pMQSvQoPPcpdWQXEYFGLkVi+WGXY5sdQEfHLwf8cMdUuVzx+WjwXEw31+hslDlq GS7Sb+LT0yYtktYyAj1WQ8Z4T6YzdsePZ5aEnhHkXlCE60V6LBujk8wHUU0l6YPd6XCK 5FCE7/eMrZdgxKGEAhrWt5l6Svr8TaGzYzVhXbDdqukmTrZknMJ9r7Niq+VlfoI1Z2l1 txe85bKYrk1IXV3rvH/aiZOxTM7fZPgT6Mr/dBwbsvgXqjO1z8KSj7LhHMdEMW0y+U5e a0BA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1718130302; x=1718735102; h=cc:to:mime-version:content-transfer-encoding:fcc:subject:date:from :references:in-reply-to:message-id:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=NV51om7aiHQeQkc031BzkWonrWwPeiG+D47N+moPbpw=; b=H+yM4XFzhp/pm4CSYSfdDE2YdtRWzHS1+9VBa6U2vJzjxjP/PxbLPrpx/mjW36Wo+Q S2seTZlJj6PFzz39YcGD1d4A2yn2aDSMTXQESpTphQBp3Jvv54QGM+lY3mSyUuMew+Gw ppBdNc1VmhFir5cSh+p4KUEPG8QKt6wYGtS+6L0Am9pYL1LtVYHRdid3FGQyvZE28Ymg bJGQrBtnH1FLO86Fo5IcWfIkyyW/FJ6bxRhlz1uXHHvrPAbN8I8jkQ23p3d/lxMikTls BDZ7Alam8rSbWG1/GwwDZDIAc0SNze72BnwwB/XX9P4X/FKO8DCzpj4GVXlDudslx5qw RuYg== X-Gm-Message-State: AOJu0YxnqUOEQvKPC0uQEDgbz0rwp5UWH5VaH372AcNK95cxVxujffCW 6rGONSovBnE7OXj7teM8RT4vCoJF8G9lPmfsz2LFFwtuXaaNgkhed9ykLA== X-Google-Smtp-Source: AGHT+IFMRuO4rW2ZiK4v2nJ6Gh0JjbLcNz9d6VOAveyvO3UHGLq4fTQtaKGgPX5VgGgLtspdzZsnsA== X-Received: by 2002:a05:600c:35c7:b0:422:48a:c04d with SMTP id 5b1f17b1804b1-422048ac3e8mr51040635e9.6.1718130302314; Tue, 11 Jun 2024 11:25:02 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-4217f633f2asm113132635e9.28.2024.06.11.11.25.01 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 11 Jun 2024 11:25:01 -0700 (PDT) Message-Id: In-Reply-To: References: Date: Tue, 11 Jun 2024 18:24:46 +0000 Subject: [PATCH 14/16] mktree: optionally add to an existing tree Fcc: Sent Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 To: git@vger.kernel.org Cc: Victoria Dye , Victoria Dye From: Victoria Dye From: Victoria Dye Allow users to specify a single "tree-ish" value as a positional argument. If provided, the contents of the given tree serve as the basis for the new tree (or trees, in --batch mode) created by 'mktree', on top of which all of the stdin-provided tree entries are applied. At a high level, the entries are "applied" to a base tree by iterating through the base tree using 'read_tree' in parallel with iterating through the sorted & deduplicated stdin entries via their iterator. That is, for each call to the 'build_index_from_tree callback of 'read_tree': * If the iterator entry precedes the base tree entry, add it to the in-core index, increment the iterator, and repeat. * If the iterator entry has the same name as the base tree entry, add the iterator entry to the index, increment the iterator, and return from the callback to continue the 'read_tree' iteration. * If the iterator entry follows the base tree entry, first check 'df_name_hash' to ensure we won't be adding an entry with the same name later (with a different mode). If there's no directory/file conflict, add the base tree entry to the index. In either case, return from the callback to continue the 'read_tree' iteration. Finally, once 'read_tree' is complete, add the remaining entries in the iterator to the index and write out the index as a tree. Signed-off-by: Victoria Dye --- Documentation/git-mktree.txt | 7 +- builtin/mktree.c | 134 ++++++++++++++++++++++++++++++----- t/t1010-mktree.sh | 36 ++++++++++ 3 files changed, 157 insertions(+), 20 deletions(-) diff --git a/Documentation/git-mktree.txt b/Documentation/git-mktree.txt index afbc846d077..99abd3c31a6 100644 --- a/Documentation/git-mktree.txt +++ b/Documentation/git-mktree.txt @@ -9,7 +9,7 @@ git-mktree - Build a tree-object from formatted tree entries SYNOPSIS -------- [verse] -'git mktree' [-z] [--missing] [--literally] [--batch] +'git mktree' [-z] [--missing] [--literally] [--batch] [--] [] DESCRIPTION ----------- @@ -40,6 +40,11 @@ OPTIONS optional. Note - if the `-z` option is used, lines are terminated with NUL. +:: + If provided, the tree entries provided in stdin are added to this tree + rather than a new empty one, replacing existing entries with identical + names. Not compatible with `--literally`. + INPUT FORMAT ------------ Tree entries may be specified in any of the formats compatible with the diff --git a/builtin/mktree.c b/builtin/mktree.c index 09b3c5c6244..9e9d2554cad 100644 --- a/builtin/mktree.c +++ b/builtin/mktree.c @@ -12,7 +12,9 @@ #include "read-cache-ll.h" #include "strbuf.h" #include "tree.h" +#include "object-name.h" #include "parse-options.h" +#include "pathspec.h" #include "object-store-ll.h" struct tree_entry { @@ -206,45 +208,122 @@ static struct tree_entry *advance_tree_entry_iterator(struct tree_entry_iterator return prev; } -static int add_tree_entry_to_index(struct index_state *istate, +struct build_index_data { + struct tree_entry_iterator iter; + struct hashmap *df_name_hash; + struct index_state istate; +}; + +static int add_tree_entry_to_index(struct build_index_data *data, struct tree_entry *ent) { struct cache_entry *ce; - struct strbuf ce_name = STRBUF_INIT; - strbuf_add(&ce_name, ent->name, ent->len); - - ce = make_cache_entry(istate, ent->mode, &ent->oid, ent->name, 0, 0); + ce = make_cache_entry(&data->istate, ent->mode, &ent->oid, ent->name, 0, 0); if (!ce) return error(_("make_cache_entry failed for path '%s'"), ent->name); - add_index_entry(istate, ce, ADD_CACHE_JUST_APPEND); - strbuf_release(&ce_name); + add_index_entry(&data->istate, ce, ADD_CACHE_JUST_APPEND); return 0; } -static void write_tree(struct tree_entry_array *arr, struct object_id *oid) +static int build_index_from_tree(const struct object_id *oid, + struct strbuf *base, const char *filename, + unsigned mode, void *context) { - struct tree_entry_iterator iter = { NULL }; + int result; + struct tree_entry *base_tree_ent; + struct build_index_data *cbdata = context; + size_t filename_len = strlen(filename); + size_t path_len = S_ISDIR(mode) ? st_add3(filename_len, base->len, 1) + : st_add(filename_len, base->len); + + /* Create a tree entry from the current entry in read_tree iteration */ + base_tree_ent = xcalloc(1, st_add3(sizeof(struct tree_entry), path_len, 1)); + base_tree_ent->len = path_len; + base_tree_ent->mode = mode; + oidcpy(&base_tree_ent->oid, oid); + + memcpy(base_tree_ent->name, base->buf, base->len); + memcpy(base_tree_ent->name + base->len, filename, filename_len); + if (S_ISDIR(mode)) + base_tree_ent->name[base_tree_ent->len - 1] = '/'; + + while (cbdata->iter.current) { + struct tree_entry *ent = cbdata->iter.current; + + int cmp = name_compare(ent->name, ent->len, + base_tree_ent->name, base_tree_ent->len); + if (!cmp || cmp < 0) { + advance_tree_entry_iterator(&cbdata->iter); + + if (add_tree_entry_to_index(cbdata, ent) < 0) { + result = error(_("failed to add tree entry '%s'"), ent->name); + goto cleanup_and_return; + } + + if (!cmp) { + result = 0; + goto cleanup_and_return; + } else + continue; + } + + break; + } + + /* + * If the tree entry should be replaced with an entry with the same name + * (but different mode), skip it. + */ + hashmap_entry_init(&base_tree_ent->ent, + memhash(base_tree_ent->name, df_path_len(base_tree_ent->len, base_tree_ent->mode))); + if (hashmap_get_entry(cbdata->df_name_hash, base_tree_ent, ent, NULL)) { + result = 0; + goto cleanup_and_return; + } + + if (add_tree_entry_to_index(cbdata, base_tree_ent)) { + result = -1; + goto cleanup_and_return; + } + + result = 0; + +cleanup_and_return: + FREE_AND_NULL(base_tree_ent); + return result; +} + +static void write_tree(struct tree_entry_array *arr, struct tree *base_tree, + struct object_id *oid) +{ + struct build_index_data cbdata = { 0 }; struct tree_entry *ent; - struct index_state istate = INDEX_STATE_INIT(the_repository); - istate.sparse_index = 1; + struct pathspec ps = { 0 }; sort_and_dedup_tree_entry_array(arr); - init_tree_entry_iterator(&iter, arr); + index_state_init(&cbdata.istate, the_repository); + cbdata.istate.sparse_index = 1; + init_tree_entry_iterator(&cbdata.iter, arr); + cbdata.df_name_hash = &arr->df_name_hash; /* Construct an in-memory index from the provided entries & base tree */ - while ((ent = advance_tree_entry_iterator(&iter))) { - if (add_tree_entry_to_index(&istate, ent)) + if (base_tree && + read_tree(the_repository, base_tree, &ps, build_index_from_tree, &cbdata) < 0) + die(_("failed to create tree")); + + while ((ent = advance_tree_entry_iterator(&cbdata.iter))) { + if (add_tree_entry_to_index(&cbdata, ent)) die(_("failed to add tree entry '%s'"), ent->name); } /* Write out new tree */ - if (cache_tree_update(&istate, WRITE_TREE_SILENT | WRITE_TREE_MISSING_OK)) + if (cache_tree_update(&cbdata.istate, WRITE_TREE_SILENT | WRITE_TREE_MISSING_OK)) die(_("failed to write tree")); - oidcpy(oid, &istate.cache_tree->oid); + oidcpy(oid, &cbdata.istate.cache_tree->oid); - release_index(&istate); + release_index(&cbdata.istate); } static void write_tree_literally(struct tree_entry_array *arr, @@ -268,7 +347,7 @@ static void write_tree_literally(struct tree_entry_array *arr, } static const char *mktree_usage[] = { - "git mktree [-z] [--missing] [--literally] [--batch]", + "git mktree [-z] [--missing] [--literally] [--batch] [--] []", NULL }; @@ -326,6 +405,7 @@ int cmd_mktree(int ac, const char **av, const char *prefix) int is_batch_mode = 0; struct tree_entry_array arr = { 0 }; struct mktree_line_data mktree_line_data = { .arr = &arr }; + struct tree *base_tree = NULL; int ret; const struct option option[] = { @@ -338,6 +418,22 @@ int cmd_mktree(int ac, const char **av, const char *prefix) }; ac = parse_options(ac, av, prefix, option, mktree_usage, 0); + if (ac > 1) + usage_with_options(mktree_usage, option); + + if (ac) { + struct object_id base_tree_oid; + + if (mktree_line_data.literally) + die(_("option '%s' and tree-ish cannot be used together"), "--literally"); + + if (repo_get_oid(the_repository, av[0], &base_tree_oid)) + die(_("not a valid object name %s"), av[0]); + + base_tree = parse_tree_indirect(&base_tree_oid); + if (!base_tree) + die(_("not a tree object: %s"), oid_to_hex(&base_tree_oid)); + } init_tree_entry_array(&arr); @@ -361,7 +457,7 @@ int cmd_mktree(int ac, const char **av, const char *prefix) if (mktree_line_data.literally) write_tree_literally(&arr, &oid); else - write_tree(&arr, &oid); + write_tree(&arr, base_tree, &oid); puts(oid_to_hex(&oid)); fflush(stdout); } diff --git a/t/t1010-mktree.sh b/t/t1010-mktree.sh index 956692347f0..ea5a011405e 100755 --- a/t/t1010-mktree.sh +++ b/t/t1010-mktree.sh @@ -238,4 +238,40 @@ test_expect_success 'mktree with duplicate entries' ' test_cmp expect actual ' +test_expect_success 'mktree with base tree' ' + tree_oid=$(cat tree) && + folder_oid=$(git rev-parse ${tree_oid}:folder) && + before_oid=$(git rev-parse ${tree_oid}:before) && + head_oid=$(git rev-parse HEAD) && + + { + printf "040000 tree $folder_oid\ttest\n" && + printf "100644 blob $before_oid\ttest.txt\n" && + printf "040000 tree $folder_oid\ttest-\n" && + printf "160000 commit $head_oid\ttest0\n" + } >top.base && + git mktree tree.base && + + { + printf "100755 blob $before_oid\tz\n" && + printf "160000 commit $head_oid\ttest.xyz\n" && + printf "040000 tree $folder_oid\ta\n" && + printf "100644 blob $before_oid\ttest\n" + } >top.append && + git mktree $(cat tree.base) tree.actual && + + { + printf "040000 tree $folder_oid\ta\n" && + printf "100644 blob $before_oid\ttest\n" && + printf "040000 tree $folder_oid\ttest-\n" && + printf "100644 blob $before_oid\ttest.txt\n" && + printf "160000 commit $head_oid\ttest.xyz\n" && + printf "160000 commit $head_oid\ttest0\n" && + printf "100755 blob $before_oid\tz\n" + } >expect && + git ls-tree $(cat tree.actual) >actual && + + test_cmp expect actual +' + test_done From patchwork Tue Jun 11 18:24:47 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Victoria Dye X-Patchwork-Id: 13694117 Received: from mail-wm1-f49.google.com (mail-wm1-f49.google.com [209.85.128.49]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 2135B8595F for ; Tue, 11 Jun 2024 18:25:05 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.49 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1718130307; cv=none; b=q+yg4ZAcQxIo0K8eKcNdxU2m/qDl4Hh2mu5nEfvgn+/7M6xW4Y1hf+XGMDpp+un8H80jUdG0Pnum096+j2wA7UzHFXBp/OeGmeW60yBlS44gbwPuGZ4UFquzZA0iKFPzjLUWQ0TdQItmrEspYkGl8j04xn8T2nxbv435kzJeI4w= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1718130307; c=relaxed/simple; bh=0IVyjfuzNt8rxSM0c/CZCQ9n39ivfh64zsuV2ro/GjE=; h=Message-Id:In-Reply-To:References:From:Date:Subject:Content-Type: MIME-Version:To:Cc; b=Vkr+CyW1W7rYynpuiDZoNgLZLOSNrpd+Fy3oEfNNWk8i1arT2Uhfg+oHWkPX6eC/07sAVCw0FS2SDM6/RHB+xZ+DprbQ7JD+PS4KDLYna2pnxRobTq+M2XDxJFw6Eoe7+8d817rTKrYJ27iq1hXXx4LALl5NGNWHkTAWZbNDr0M= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=ATjfvY1Z; arc=none smtp.client-ip=209.85.128.49 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="ATjfvY1Z" Received: by mail-wm1-f49.google.com with SMTP id 5b1f17b1804b1-42189d3c7efso25889725e9.2 for ; Tue, 11 Jun 2024 11:25:04 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1718130303; x=1718735103; darn=vger.kernel.org; h=cc:to:mime-version:content-transfer-encoding:fcc:subject:date:from :references:in-reply-to:message-id:from:to:cc:subject:date :message-id:reply-to; bh=NMUgIifCil1XQvF8cIEvY0+SJAPRnU7WEq//C93eiBY=; b=ATjfvY1ZjJ1CI6B+UFiqhSU0YqGpR7aBKSwfO1w12oPO14PuDEKKMu99bxtpAOg6ik J9DOxCXIh4JkonPeANh6tdZYKVW02bKAJAQUqJ6yHfZtS71m5sog97AjBDz7+t/ZSxZ3 Dghx2vmu7UdOvYJMPRi2GYW591ZQtFXwGdWk/AJMZojNWhC70qn5GjwJ6q620vC55PZl 69DwzUSy9/3E3iZkQeiRb87JQKdrm7tDQ4MOIfZArNJCW/1JQ1FKhJq0KkBkQru5dSrP tGc/NXnoRSUgrGg8Y9FwAoosEnEiIevIaMNsbKFNnYcsSuZGHwIZwq3Mk9U3vxJCxYM3 XHzg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1718130303; x=1718735103; h=cc:to:mime-version:content-transfer-encoding:fcc:subject:date:from :references:in-reply-to:message-id:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=NMUgIifCil1XQvF8cIEvY0+SJAPRnU7WEq//C93eiBY=; b=oHCkKv6XLwrB3D0Sv/whSd8tOn3vJo8R9dHEXhEVQSXnVxD7ajEpF9CYyGwoXK9ppa PqXDRvc4el2Tg0wCCSuRyMPoa5qrkAiXdtqHgjej2T3f86LyPG7/vTpcwQq9Hj4roUvj RIF860Kn43D5L1sGWbEpRrqilkJ2bVs0dFnSrInlSh7wTeRNmKFeMqjl6Ui9KXE0af21 e54ePEWywVZJarGUpcT/OkSHLGQMFS8eiyTqY9DbzNCygjKcf6FPn1fIm4DXgR7o8zsZ J48tNq1IDsmvFR0t6buQi/XBZiWrvO7IAeILnbdMQuum7Mq8+O6yFexuJGI58A8oTH7A vEEQ== X-Gm-Message-State: AOJu0YwvvuXus9iPjelfZVD3cDEV2A4SwMOHCsQ35u5qD//M9/N7a4Tk JGvPPbm5AyMV4M0WYqtbDNN9irYaZywg5BDwsrLmudzgmj+naR7RMWrsDQ== X-Google-Smtp-Source: AGHT+IEFNizW+ECTuQqUy3hJ1PoqvU5o9nnPjfS/WCSrWj7jrHsQq1J1W3jvZm0zWVz/9QCnASpWGA== X-Received: by 2002:a05:6000:367:b0:357:7590:921d with SMTP id ffacd0b85a97d-35efee184f5mr10825665f8f.66.1718130302999; Tue, 11 Jun 2024 11:25:02 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-35f23a651d4sm6430408f8f.50.2024.06.11.11.25.02 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 11 Jun 2024 11:25:02 -0700 (PDT) Message-Id: <058354f45f7b837ebeb08337a8dfd6e0ec1e9d1b.1718130288.git.gitgitgadget@gmail.com> In-Reply-To: References: Date: Tue, 11 Jun 2024 18:24:47 +0000 Subject: [PATCH 15/16] mktree: allow deeper paths in input Fcc: Sent Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 To: git@vger.kernel.org Cc: Victoria Dye , Victoria Dye From: Victoria Dye From: Victoria Dye Update 'git mktree' to handle entries nested inside of directories (e.g. 'path/to/a/file.txt'). This functionality requires a series of changes: * In 'sort_and_dedup_tree_entry_array()', remove entries inside of directories that come after them in input order. * Also in 'sort_and_dedup_tree_entry_array()', mark directories that contain entries that come after them in input order (e.g., 'folder/' followed by 'folder/file.txt') as "need to expand". * In 'add_tree_entry_to_index()', if a tree entry is marked as "need to expand", recurse into it with 'read_tree_at()' & 'build_index_from_tree'. * In 'build_index_from_tree()', if a user-specified tree entry is contained within the current iterated entry, return 'READ_TREE_RECURSIVE' to recurse into the iterated tree. Signed-off-by: Victoria Dye --- Documentation/git-mktree.txt | 5 ++ builtin/mktree.c | 101 ++++++++++++++++++++++++++++++--- t/t1010-mktree.sh | 107 +++++++++++++++++++++++++++++++++-- 3 files changed, 200 insertions(+), 13 deletions(-) diff --git a/Documentation/git-mktree.txt b/Documentation/git-mktree.txt index 99abd3c31a6..db90fdcdc8f 100644 --- a/Documentation/git-mktree.txt +++ b/Documentation/git-mktree.txt @@ -50,6 +50,11 @@ INPUT FORMAT Tree entries may be specified in any of the formats compatible with the `--index-info` option to linkgit:git-update-index[1]. +Entries may use full pathnames containing directory separators to specify +entries nested within one or more directories. These entries are inserted into +the appropriate tree in the base tree-ish if one exists. Otherwise, empty parent +trees are created to contain the entries. + The order of the tree entries is normalized by `mktree` so pre-sorting the input by path is not required. Multiple entries provided with the same path are deduplicated, with only the last one specified added to the tree. diff --git a/builtin/mktree.c b/builtin/mktree.c index 9e9d2554cad..00b77869a56 100644 --- a/builtin/mktree.c +++ b/builtin/mktree.c @@ -22,6 +22,7 @@ struct tree_entry { /* Internal */ size_t order; + int expand_dir; unsigned mode; struct object_id oid; @@ -39,6 +40,7 @@ struct tree_entry_array { struct tree_entry **entries; struct hashmap df_name_hash; + int has_nested_entries; }; static int df_name_hash_cmp(const void *cmp_data UNUSED, @@ -70,6 +72,13 @@ static void tree_entry_array_push(struct tree_entry_array *arr, struct tree_entr arr->entries[arr->nr++] = ent; } +static struct tree_entry *tree_entry_array_pop(struct tree_entry_array *arr) +{ + if (!arr->nr) + return NULL; + return arr->entries[--arr->nr]; +} + static void clear_tree_entry_array(struct tree_entry_array *arr) { hashmap_clear(&arr->df_name_hash); @@ -107,8 +116,10 @@ static void append_to_tree(unsigned mode, struct object_id *oid, const char *pat if (!verify_path(ent->name, mode)) die(_("invalid path '%s'"), path); - if (strchr(ent->name, '/')) - die("path %s contains slash", path); + + /* mark has_nested_entries if needed */ + if (!arr->has_nested_entries && strchr(ent->name, '/')) + arr->has_nested_entries = 1; /* Add trailing slash to dir */ if (S_ISDIR(mode)) @@ -167,6 +178,46 @@ static void sort_and_dedup_tree_entry_array(struct tree_entry_array *arr) ignore_mode = 0; QSORT_S(arr->entries, arr->nr, ent_compare, &ignore_mode); + if (arr->has_nested_entries) { + struct tree_entry_array parent_dir_ents = { 0 }; + + count = arr->nr; + arr->nr = 0; + + /* Remove any entries where one of its parent dirs has a higher 'order' */ + for (size_t i = 0; i < count; i++) { + const char *skipped_prefix; + struct tree_entry *parent; + struct tree_entry *curr = arr->entries[i]; + int skip_entry = 0; + + while ((parent = tree_entry_array_pop(&parent_dir_ents))) { + if (!skip_prefix(curr->name, parent->name, &skipped_prefix)) + continue; + + /* entry in dir, so we push the parent back onto the stack */ + tree_entry_array_push(&parent_dir_ents, parent); + + if (parent->order > curr->order) + skip_entry = 1; + else + parent->expand_dir = 1; + + break; + } + + if (!skip_entry) { + arr->entries[arr->nr++] = curr; + if (S_ISDIR(curr->mode)) + tree_entry_array_push(&parent_dir_ents, curr); + } else { + FREE_AND_NULL(curr); + } + } + + release_tree_entry_array(&parent_dir_ents); + } + /* Finally, initialize the directory-file conflict hash map */ for (size_t i = 0; i < count; i++) { struct tree_entry *curr = arr->entries[i]; @@ -214,15 +265,40 @@ struct build_index_data { struct index_state istate; }; +static int build_index_from_tree(const struct object_id *oid, + struct strbuf *base, const char *filename, + unsigned mode, void *context); + static int add_tree_entry_to_index(struct build_index_data *data, struct tree_entry *ent) { - struct cache_entry *ce; - ce = make_cache_entry(&data->istate, ent->mode, &ent->oid, ent->name, 0, 0); - if (!ce) - return error(_("make_cache_entry failed for path '%s'"), ent->name); + if (ent->expand_dir) { + int ret = 0; + struct pathspec ps = { 0 }; + struct tree *subtree = parse_tree_indirect(&ent->oid); + struct strbuf base_path = STRBUF_INIT; + strbuf_add(&base_path, ent->name, ent->len); + + if (!subtree) + ret = error(_("not a tree object: %s"), oid_to_hex(&ent->oid)); + else if (read_tree_at(the_repository, subtree, &base_path, 0, &ps, + build_index_from_tree, data) < 0) + ret = -1; + + strbuf_release(&base_path); + if (ret) + return ret; + + } else { + struct cache_entry *ce = make_cache_entry(&data->istate, + ent->mode, &ent->oid, + ent->name, 0, 0); + if (!ce) + return error(_("make_cache_entry failed for path '%s'"), ent->name); + + add_index_entry(&data->istate, ce, ADD_CACHE_JUST_APPEND); + } - add_index_entry(&data->istate, ce, ADD_CACHE_JUST_APPEND); return 0; } @@ -249,10 +325,12 @@ static int build_index_from_tree(const struct object_id *oid, base_tree_ent->name[base_tree_ent->len - 1] = '/'; while (cbdata->iter.current) { + const char *skipped_prefix; struct tree_entry *ent = cbdata->iter.current; + int cmp; - int cmp = name_compare(ent->name, ent->len, - base_tree_ent->name, base_tree_ent->len); + cmp = name_compare(ent->name, ent->len, + base_tree_ent->name, base_tree_ent->len); if (!cmp || cmp < 0) { advance_tree_entry_iterator(&cbdata->iter); @@ -266,6 +344,11 @@ static int build_index_from_tree(const struct object_id *oid, goto cleanup_and_return; } else continue; + } else if (skip_prefix(ent->name, base_tree_ent->name, &skipped_prefix) && + S_ISDIR(base_tree_ent->mode)) { + /* The entry is in the current traversed tree entry, so we recurse */ + result = READ_TREE_RECURSIVE; + goto cleanup_and_return; } break; diff --git a/t/t1010-mktree.sh b/t/t1010-mktree.sh index ea5a011405e..1d6365141fc 100755 --- a/t/t1010-mktree.sh +++ b/t/t1010-mktree.sh @@ -89,12 +89,21 @@ test_expect_success 'mktree with invalid submodule OIDs' ' grep "object $tree_oid is a tree but specified type was (commit)" err ' -test_expect_success 'mktree refuses to read ls-tree -r output (1)' ' - test_must_fail git mktree actual && + test_cmp tree actual ' -test_expect_success 'mktree refuses to read ls-tree -r output (2)' ' - test_must_fail git mktree actual && + test_cmp tree.withsub actual +' + +test_expect_success 'mktree de-duplicates files inside directories' ' + git ls-tree $(cat tree) >everything && + cat top_and_all && + git mktree actual && + test_cmp tree actual ' test_expect_success 'mktree fails on malformed input' ' @@ -238,6 +247,50 @@ test_expect_success 'mktree with duplicate entries' ' test_cmp expect actual ' +test_expect_success 'mktree adds entry after nested entry' ' + tree_oid=$(cat tree) && + folder_oid=$(git rev-parse ${tree_oid}:folder) && + one_oid=$(git rev-parse ${tree_oid}:folder/one) && + + { + printf "040000 tree $folder_oid\tearly\n" && + printf "100644 blob $one_oid\tearly/one\n" && + printf "100644 blob $one_oid\tlater\n" && + printf "040000 tree $EMPTY_TREE\tnew-tree\n" && + printf "100644 blob $one_oid\tnew-tree/one\n" && + printf "100644 blob $one_oid\tzzz\n" + } >top.rec && + git mktree tree.actual && + + { + printf "040000 tree $folder_oid\tearly\n" && + printf "100644 blob $one_oid\tlater\n" && + printf "040000 tree $folder_oid\tnew-tree\n" && + printf "100644 blob $one_oid\tzzz\n" + } >expect && + git ls-tree $(cat tree.actual) >actual && + + test_cmp expect actual +' + +test_expect_success 'mktree inserts entries into directories' ' + folder_oid=$(git rev-parse ${tree_oid}:folder) && + one_oid=$(git rev-parse ${tree_oid}:folder/one) && + blob_oid=$(git rev-parse ${tree_oid}:before) && + { + printf "040000 tree $folder_oid\tfolder\n" && + printf "100644 blob $blob_oid\tfolder/two\n" + } | git mktree >actual && + + { + printf "100644 blob $one_oid\tfolder/one\n" && + printf "100644 blob $blob_oid\tfolder/two\n" + } >expect && + git ls-tree -r $(cat actual) >actual && + + test_cmp expect actual +' + test_expect_success 'mktree with base tree' ' tree_oid=$(cat tree) && folder_oid=$(git rev-parse ${tree_oid}:folder) && @@ -274,4 +327,50 @@ test_expect_success 'mktree with base tree' ' test_cmp expect actual ' +test_expect_success 'mktree with base tree (deep)' ' + tree_oid=$(cat tree) && + folder_oid=$(git rev-parse ${tree_oid}:folder) && + before_oid=$(git rev-parse ${tree_oid}:before) && + folder_one_oid=$(git rev-parse ${tree_oid}:folder/one) && + head_oid=$(git rev-parse HEAD) && + + { + printf "100755 blob $before_oid\tfolder/before\n" && + printf "100644 blob $before_oid\tfolder/one.txt\n" && + printf "160000 commit $head_oid\tfolder/sub\n" && + printf "040000 tree $folder_oid\tfolder/one\n" && + printf "040000 tree $folder_oid\tfolder/one/deeper\n" + } >top.append && + git mktree tree.actual && + + { + printf "100755 blob $before_oid\tfolder/before\n" && + printf "100644 blob $before_oid\tfolder/one.txt\n" && + printf "100644 blob $folder_one_oid\tfolder/one/deeper/one\n" && + printf "100644 blob $folder_one_oid\tfolder/one/one\n" && + printf "160000 commit $head_oid\tfolder/sub\n" + } >expect && + git ls-tree -r $(cat tree.actual) -- folder/ >actual && + + test_cmp expect actual +' + +test_expect_success 'mktree fails on directory-file conflict' ' + tree_oid="$(cat tree)" && + blob_oid="$(git rev-parse $tree_oid:folder.txt)" && + + { + printf "100644 blob $blob_oid\ttest\n" && + printf "100644 blob $blob_oid\ttest/deeper\n" + } | + test_must_fail git mktree 2>err && + grep "You have both test and test/deeper" err && + + { + printf "100644 blob $blob_oid\tfolder/one/deeper/deep\n" + } | + test_must_fail git mktree $tree_oid 2>err && + grep "You have both folder/one and folder/one/deeper/deep" err +' + test_done From patchwork Tue Jun 11 18:24:48 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Victoria Dye X-Patchwork-Id: 13694118 Received: from mail-wm1-f54.google.com (mail-wm1-f54.google.com [209.85.128.54]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 9E7FA13210D for ; Tue, 11 Jun 2024 18:25:05 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.54 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1718130307; cv=none; b=FBx5FDgUppcJq51FgEp/o6Pd6tYDKgEZgRFWwPtFGqw8HzCQbJdQ+E4tAXVnRLkg5RXiUKlLXJxdN/9joyRzNjlAGGPmrfazBKMOzWCMv9rjVLqAkehY1Gk5KNVM0kyrDxz21+ojIbCxecW/2uv55FywSJmZCPNYOVHe0quUYxI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1718130307; c=relaxed/simple; bh=03OzM0oh5ra+ghw1/SmdZwx+xZMkipokS8ZxnLHydiE=; h=Message-Id:In-Reply-To:References:From:Date:Subject:Content-Type: MIME-Version:To:Cc; b=hDfP3cw9qjOOxqm7QeIlQ/8a2xEtcrS3+NtVVUjg2rOW4mFV4xdNurLEla5OG0u8Y2O9li9KhvvUbUp8sDavT9qDZpM54zmllOzdyU/ehsCVwTk7sONX3vbcMMN/1674TTmwB/XgYjYVF0NdTwwKqy4KZQ1L3QlCeZbnQgAAfUg= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=aPB+Ut97; arc=none smtp.client-ip=209.85.128.54 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="aPB+Ut97" Received: by mail-wm1-f54.google.com with SMTP id 5b1f17b1804b1-4210aa012e5so14794245e9.0 for ; Tue, 11 Jun 2024 11:25:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1718130304; x=1718735104; darn=vger.kernel.org; h=cc:to:mime-version:content-transfer-encoding:fcc:subject:date:from :references:in-reply-to:message-id:from:to:cc:subject:date :message-id:reply-to; bh=D+X6OiwowJLdCbJOKgYcMkQVtYykLJ8vrOMNLn5YdrM=; b=aPB+Ut978b9+NiDFfmDGHuYEYFyALyRfLNvuvWKncIjfm4VDF2+UO1m/pEqNQ9cX1J myWGauRbmE1ViSzYg1djj4OU+9XGP+NL0JZVmph7p1B7YHHSTIJkzixBXaWVp9kKhGdj bw2Nra+ABQD/1hA7OXbnKWEfRRmVIm+FSzSjx4uBdyxyU+julq16x3H3P/9u2R1fwkxS x7hoepEF6JBHVKovmAzV26MAMfU9sD2XQOPqZE5vlssNceG9x+SQbVz7YmYu4tchy/eP jPOON2IzF8yIeiwYSjdwJ17wzpNvTrqzKG253jdvGFE05VkTcS3e6mvk6qBu83FsYWNv 49iA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1718130304; x=1718735104; h=cc:to:mime-version:content-transfer-encoding:fcc:subject:date:from :references:in-reply-to:message-id:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=D+X6OiwowJLdCbJOKgYcMkQVtYykLJ8vrOMNLn5YdrM=; b=OEsm3ZjfaWQd0ZcsUli+HDhI1K3Tg1pKG1+D9+xh16pu59V6bnxKAUtHg6hQa/W4JV LTH+oo0I99NjZlqS8y8mnsxxf0wKrKA36z5mjXtg+W40V2SEkn21C/uig4jnrSCX6pZh 3eANUdL9oMjvhJmFNmxEP9tn3EOWFdp09KUn9P8yWRL9izFXdWLM6wC9WqS+iB1UHDm0 INfRLLtBI8EgXnSrvRE4bxiTWsw7yWen80fV8YZxBZiOcrGAbG+n2kD7c6nI3csjcicH 66quNksPD0NDSd9Z45z//AYEV8rNuyfHIxqIl7J2Ht4hhJY48Fc2WrBmqXnGdiY8ei6W LPZg== X-Gm-Message-State: AOJu0Yxlrvi4KVB/xKfEaPFzuMVpUeqpOlqHG7vFSPSy+khNkn2lyb+L BA9tguYKtdGTXb8ooDPvmLg4E0XycR7WOxR1u4ezeRlRHSavYhfq2Enhdw== X-Google-Smtp-Source: AGHT+IHL4tupzGUuW5pJMpitAPNpDtLHHdj4yt1gO9raPorMvZhvJIbAnDGc159v2x2hC1XEJIiaWw== X-Received: by 2002:a05:600c:1f06:b0:421:9502:3f24 with SMTP id 5b1f17b1804b1-42195023fe4mr64786115e9.14.1718130303658; Tue, 11 Jun 2024 11:25:03 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-4215c2c690dsm189587455e9.34.2024.06.11.11.25.03 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 11 Jun 2024 11:25:03 -0700 (PDT) Message-Id: In-Reply-To: References: Date: Tue, 11 Jun 2024 18:24:48 +0000 Subject: [PATCH 16/16] mktree: remove entries when mode is 0 Fcc: Sent Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 To: git@vger.kernel.org Cc: Victoria Dye , Victoria Dye From: Victoria Dye From: Victoria Dye If tree entries are specified with a mode with value '0', remove them from the tree instead of adding/updating them. If the mode is '0', both the provided type string (if specified) and the object ID of the entry are ignored. Note that entries with mode '0' are added to the 'struct tree_ent_array' with a trailing slash so that it's always treated like a directory. This is a bit of a hack to ensure that the removal supercedes any preceding entries with matching names, as well as any nested inside a directory matching its name. Signed-off-by: Victoria Dye --- Documentation/git-mktree.txt | 4 +++ builtin/mktree.c | 64 ++++++++++++++++++++---------------- t/t1010-mktree.sh | 38 +++++++++++++++++++++ 3 files changed, 77 insertions(+), 29 deletions(-) diff --git a/Documentation/git-mktree.txt b/Documentation/git-mktree.txt index db90fdcdc8f..a660438c67f 100644 --- a/Documentation/git-mktree.txt +++ b/Documentation/git-mktree.txt @@ -55,6 +55,10 @@ entries nested within one or more directories. These entries are inserted into the appropriate tree in the base tree-ish if one exists. Otherwise, empty parent trees are created to contain the entries. +An entry with a mode of "0" will remove an entry of the same name from the base +tree-ish. If no tree-ish argument is given, or the entry does not exist in that +tree, the entry is ignored. + The order of the tree entries is normalized by `mktree` so pre-sorting the input by path is not required. Multiple entries provided with the same path are deduplicated, with only the last one specified added to the tree. diff --git a/builtin/mktree.c b/builtin/mktree.c index 00b77869a56..e94c9ca7e87 100644 --- a/builtin/mktree.c +++ b/builtin/mktree.c @@ -32,7 +32,7 @@ struct tree_entry { static inline size_t df_path_len(size_t pathlen, unsigned int mode) { - return S_ISDIR(mode) ? pathlen - 1 : pathlen; + return (S_ISDIR(mode) || !mode) ? pathlen - 1 : pathlen; } struct tree_entry_array { @@ -106,7 +106,7 @@ static void append_to_tree(unsigned mode, struct object_id *oid, const char *pat size_t len_to_copy = len; /* Normalize and validate entry path */ - if (S_ISDIR(mode)) { + if (S_ISDIR(mode) || !mode) { while(len_to_copy > 0 && is_dir_sep(path[len_to_copy - 1])) len_to_copy--; len = len_to_copy + 1; /* add space for trailing slash */ @@ -122,7 +122,7 @@ static void append_to_tree(unsigned mode, struct object_id *oid, const char *pat arr->has_nested_entries = 1; /* Add trailing slash to dir */ - if (S_ISDIR(mode)) + if (S_ISDIR(mode) || !mode) ent->name[len - 1] = '/'; } @@ -208,7 +208,7 @@ static void sort_and_dedup_tree_entry_array(struct tree_entry_array *arr) if (!skip_entry) { arr->entries[arr->nr++] = curr; - if (S_ISDIR(curr->mode)) + if (S_ISDIR(curr->mode) || !curr->mode) tree_entry_array_push(&parent_dir_ents, curr); } else { FREE_AND_NULL(curr); @@ -272,6 +272,9 @@ static int build_index_from_tree(const struct object_id *oid, static int add_tree_entry_to_index(struct build_index_data *data, struct tree_entry *ent) { + if (!ent->mode) + return 0; + if (ent->expand_dir) { int ret = 0; struct pathspec ps = { 0 }; @@ -445,36 +448,39 @@ static int mktree_line(unsigned int mode, struct object_id *oid, const char *path, void *cbdata) { struct mktree_line_data *data = cbdata; - enum object_type mode_type = object_type(mode); - struct object_info oi = OBJECT_INFO_INIT; - enum object_type parsed_obj_type; - if (obj_type && mode_type != obj_type) - die("object type (%s) doesn't match mode type (%s)", - type_name(obj_type), type_name(mode_type)); + if (mode) { + struct object_info oi = OBJECT_INFO_INIT; + enum object_type parsed_obj_type; + enum object_type mode_type = object_type(mode); - oi.typep = &parsed_obj_type; + if (obj_type && mode_type != obj_type) + die("object type (%s) doesn't match mode type (%s)", + type_name(obj_type), type_name(mode_type)); - if (oid_object_info_extended(the_repository, oid, &oi, - OBJECT_INFO_LOOKUP_REPLACE | - OBJECT_INFO_QUICK | - OBJECT_INFO_SKIP_FETCH_OBJECT) < 0) - parsed_obj_type = -1; + oi.typep = &parsed_obj_type; - if (parsed_obj_type < 0) { - if (data->allow_missing || S_ISGITLINK(mode)) { - ; /* no problem - missing objects & submodules are presumed to be of the right type */ - } else { - die("entry '%s' object %s is unavailable", path, oid_to_hex(oid)); + if (oid_object_info_extended(the_repository, oid, &oi, + OBJECT_INFO_LOOKUP_REPLACE | + OBJECT_INFO_QUICK | + OBJECT_INFO_SKIP_FETCH_OBJECT) < 0) + parsed_obj_type = -1; + + if (parsed_obj_type < 0) { + if (data->allow_missing || S_ISGITLINK(mode)) { + ; /* no problem - missing objects & submodules are presumed to be of the right type */ + } else { + die("entry '%s' object %s is unavailable", path, oid_to_hex(oid)); + } + } else if (parsed_obj_type != mode_type) { + /* + * The object exists but is of the wrong type. + * This is a problem regardless of allow_missing + * because the new tree entry will never be correct. + */ + die("entry '%s' object %s is a %s but specified type was (%s)", + path, oid_to_hex(oid), type_name(parsed_obj_type), type_name(mode_type)); } - } else if (parsed_obj_type != mode_type) { - /* - * The object exists but is of the wrong type. - * This is a problem regardless of allow_missing - * because the new tree entry will never be correct. - */ - die("entry '%s' object %s is a %s but specified type was (%s)", - path, oid_to_hex(oid), type_name(parsed_obj_type), type_name(mode_type)); } append_to_tree(mode, oid, path, data->arr, data->literally); diff --git a/t/t1010-mktree.sh b/t/t1010-mktree.sh index 1d6365141fc..7cb88e32d4f 100755 --- a/t/t1010-mktree.sh +++ b/t/t1010-mktree.sh @@ -373,4 +373,42 @@ test_expect_success 'mktree fails on directory-file conflict' ' grep "You have both folder/one and folder/one/deeper/deep" err ' +test_expect_success 'mktree with remove entries' ' + tree_oid="$(cat tree)" && + blob_oid="$(git rev-parse $tree_oid:folder.txt)" && + + { + printf "100644 blob $blob_oid\ttest/deeper/deep.txt\n" && + printf "100644 blob $blob_oid\ttest.txt\n" && + printf "100644 blob $blob_oid\texample\n" && + printf "100644 blob $blob_oid\texample.a/file\n" && + printf "100644 blob $blob_oid\texample.txt\n" && + printf "040000 tree $tree_oid\tfolder\n" && + printf "0 $ZERO_OID\tfolder\n" && + printf "0 $ZERO_OID\tmissing\n" + } | git mktree >tree.base && + + { + printf "0 $ZERO_OID\texample.txt\n" && + printf "0 $ZERO_OID\ttest/deeper\n" + } | git mktree $(cat tree.base) >tree.actual && + + { + printf "100644 blob $blob_oid\texample\n" && + printf "100644 blob $blob_oid\texample.a/file\n" && + printf "100644 blob $blob_oid\ttest.txt\n" + } >expect && + git ls-tree -r $(cat tree.actual) >actual && + + test_cmp expect actual +' + +test_expect_success 'type and oid not checked if entry mode is 0' ' + # type and oid do not match + printf "0 commit $EMPTY_TREE\tfolder.txt\n" | + git mktree >tree.actual && + + test "$(cat tree.actual)" = $EMPTY_TREE +' + test_done