From patchwork Fri Apr 12 23:04:39 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Phillip Wood via GitGitGadget X-Patchwork-Id: 10899217 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id E91A61800 for ; Fri, 12 Apr 2019 23:04:43 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id C50932874C for ; Fri, 12 Apr 2019 23:04:43 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id B96922890F; Fri, 12 Apr 2019 23:04:43 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 5AFFE2874C for ; Fri, 12 Apr 2019 23:04:43 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726898AbfDLXEm (ORCPT ); Fri, 12 Apr 2019 19:04:42 -0400 Received: from mail-ed1-f65.google.com ([209.85.208.65]:33811 "EHLO mail-ed1-f65.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726757AbfDLXEl (ORCPT ); Fri, 12 Apr 2019 19:04:41 -0400 Received: by mail-ed1-f65.google.com with SMTP id x14so9780552eds.1 for ; Fri, 12 Apr 2019 16:04:40 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=date:message-id:in-reply-to:references:from:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=BsEVxwyGyoDVVV5NEmNsZmb1rv6alVm83kw4OD5W1pA=; b=S8FU4lfGAvhS6e/P+Lz+D6j5m0ySI9nVGsU0TlURQq3ixlC1690O6o9gW1R8ClARhT c/FSot311/mDCYg0QK4OJkKgfOD2kgZKCjFdhb5am4Iy81B5pFEArBWIlOW2f8YngA+p KAHjZtn67mdOXDztrFlo65rcAwUYnM4yd8KHe5dngWAWEC41blGo7am7vKUy3c9eQCtc TAFo8Zc8y3fWp/1CPHHwD6ow5AMkJtqvqGybz0pHVn49g8fqW6jbP4C/6ZqP1QmCsnnS k3kudZhe1SZjEM3YWSqZKMl1jnWIHN616AHkBjojWi1oKNnaRKftBc/NoA8JipdJ57+I LCSw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:message-id:in-reply-to:references:from :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=BsEVxwyGyoDVVV5NEmNsZmb1rv6alVm83kw4OD5W1pA=; b=tUpv5gTMdGwwrNeVB13LushdX3C1ess9UVlpgKCdPBLFOozmRfAgh/N7lN61UYqqLp fGvGoe6lCBCJXV6etf1GrIfOzkeUxHHvLzItJJa4CslRMjARdj3c5KBfVP03ScSTVi7p +ef9JtMGTr1byoPBExqv1dhClyFNsrDC+LcGVxSYNl/UoXSe+cd/HZffT8TxET1dreKw ON4Fgt/2XGitOii5nrXylc/sXC2b56MDOrJvHbapGsGpZNuNrRHps+gtjzG2/PZ+m61z Ua1zAvsE+CIsMQnsKpDnbwLepITOVDmljHvMcQC9RHF1wLpZXb0ywRTWTu0Ycbp4/1Dr O1lw== X-Gm-Message-State: APjAAAV3D0B+F2MVmZ6QtaszmTmQjmhADGlAIpy0wV0tbduDTkTv8A5m Mrm2DMAXlErDu21mg+HU+aYXs9GA X-Google-Smtp-Source: APXvYqwfcw8vAUwi3BS4pFeRA81h36AXPioIP2pPu4HphnMjKnY74kk23bNuKwlVWyCPc26LGvLo+A== X-Received: by 2002:a50:86f0:: with SMTP id 45mr11442874edu.153.1555110280199; Fri, 12 Apr 2019 16:04:40 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id i9sm6058830edr.76.2019.04.12.16.04.39 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 12 Apr 2019 16:04:39 -0700 (PDT) Date: Fri, 12 Apr 2019 16:04:39 -0700 (PDT) X-Google-Original-Date: Fri, 12 Apr 2019 23:04:37 GMT Message-Id: <7a9525a78a7b7b237150b9264cf675a4a0b37267.1555110278.git.gitgitgadget@gmail.com> In-Reply-To: References: From: "Rohit Ashiwal via GitGitGadget" Subject: [PATCH 1/2] archive: replace write_or_die() calls with write_block_or_die() Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: Junio C Hamano , Rohit Ashiwal Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Rohit Ashiwal MinGit for Windows comes without `gzip` bundled inside, git-archive uses `gzip -cn` to compress tar files but for this to work, gzip needs to be present on the host system. In the next commit, we will change the gzip compression so that we no longer spawn `gzip` but let zlib perform the compression in the same process instead. In preparation for this, we consolidate all the block writes into a single function. This closes https://github.com/git-for-windows/git/issues/1970 Signed-off-by: Rohit Ashiwal Signed-off-by: Johannes Schindelin --- archive-tar.c | 20 ++++++++++++++++---- 1 file changed, 16 insertions(+), 4 deletions(-) diff --git a/archive-tar.c b/archive-tar.c index 4aabd566fb..ba37dad27c 100644 --- a/archive-tar.c +++ b/archive-tar.c @@ -17,6 +17,8 @@ static unsigned long offset; static int tar_umask = 002; +static gzFile gzip; + static int write_tar_filter_archive(const struct archiver *ar, struct archiver_args *args); @@ -38,11 +40,21 @@ static int write_tar_filter_archive(const struct archiver *ar, #define USTAR_MAX_MTIME 077777777777ULL #endif +/* writes out the whole block, or dies if fails */ +static void write_block_or_die(const char *block) { + if (gzip) { + if (gzwrite(gzip, block, (unsigned) BLOCKSIZE) != BLOCKSIZE) + die(_("gzwrite failed")); + } else { + write_or_die(1, block, BLOCKSIZE); + } +} + /* writes out the whole block, but only if it is full */ static void write_if_needed(void) { if (offset == BLOCKSIZE) { - write_or_die(1, block, BLOCKSIZE); + write_block_or_die(block); offset = 0; } } @@ -66,7 +78,7 @@ static void do_write_blocked(const void *data, unsigned long size) write_if_needed(); } while (size >= BLOCKSIZE) { - write_or_die(1, buf, BLOCKSIZE); + write_block_or_die(buf); size -= BLOCKSIZE; buf += BLOCKSIZE; } @@ -101,10 +113,10 @@ static void write_trailer(void) { int tail = BLOCKSIZE - offset; memset(block + offset, 0, tail); - write_or_die(1, block, BLOCKSIZE); + write_block_or_die(block); if (tail < 2 * RECORDSIZE) { memset(block, 0, offset); - write_or_die(1, block, BLOCKSIZE); + write_block_or_die(block); } } From patchwork Fri Apr 12 23:04:40 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Phillip Wood via GitGitGadget X-Patchwork-Id: 10899221 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 81EFF17E1 for ; Fri, 12 Apr 2019 23:04:45 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 65158280FC for ; Fri, 12 Apr 2019 23:04:45 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 59BDE288BB; Fri, 12 Apr 2019 23:04:45 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id E3C73280FC for ; Fri, 12 Apr 2019 23:04:44 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726920AbfDLXEo (ORCPT ); Fri, 12 Apr 2019 19:04:44 -0400 Received: from mail-ed1-f65.google.com ([209.85.208.65]:45898 "EHLO mail-ed1-f65.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726477AbfDLXEm (ORCPT ); Fri, 12 Apr 2019 19:04:42 -0400 Received: by mail-ed1-f65.google.com with SMTP id o26so9719708edv.12 for ; Fri, 12 Apr 2019 16:04:41 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=date:message-id:in-reply-to:references:from:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=U1ehrQbrrO4btWyEjYC6AvboSluzq0Fx8j3ErJpjCAE=; b=ddl60htpHXNLWzz0F+14HFRsBw4FQBf0RBJlF/bLiGWBAn0sypGHMUu+jhLl2FVfKH TIPaLXVOMpajYJ+nks9AGicHcx7jsWqzWCSPHrwWq3X98X2kpbHY2XoHRBoiXGheoizm vzs8jJwlYzb8+wZkxIT7GziuFO9pkdbJN7F+juEVoNvfrroJBf2H9haYDtTqc4TKTjjq Ve1IS8KLXcKwbYn178Y9HmnAviafRgtWWYWbZWPAuN4SHFf0WsX13hG9WPNPopkTB5db 4wtg71kTs786iKjticdyKgFiSoItbGiH5N7bn7wQBbdEMEAJBqHFvl+KwB5r3iUaIT/U Y0EA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:message-id:in-reply-to:references:from :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=U1ehrQbrrO4btWyEjYC6AvboSluzq0Fx8j3ErJpjCAE=; b=beccLoCS5I/42VZEUPRvIXMUGbbBHjC2XCOSDWPdT7uwPYYGICvqGhxWm/9/2fHOsD YdMFW/ZAyXXhXNeF08uEnTNEmbO67uZaNjZ6vC1isUvJR2rb2Nm1GFxV+bC1qsfZTtlB 8bJtifM4JLQnqxlSV3ThKHWP+UQgCjBYQrGQbXpL7aW/eEepDBOcdZjtj8hGI9hO6frC iSB6MA+/KJSfbNoJQXsNJgBvJ/fdWVzKD4rJIDKUKjhS80Fx2G/Xe5YRHMLnxxxC0XnJ 2sg25p856T4iFdDTPpfUYN3TzwntJnQzhNmEmM7VVtZvBITvuRRHwFGT2mFkeipB3tfB 2x6A== X-Gm-Message-State: APjAAAUh0lWorMForHPCPpvl6FeiA9BoghtUj4qtZr7kibvkE8XZvg7/ aS6MJwg4Di+jizDhNBpaflReTfkA X-Google-Smtp-Source: APXvYqxHF3ERFjdc95b0V7bS8xDu62GM3LUk0oiCP3mLGOBvSHIsA1RRIHmNBoG6WSZleyI/X6cb+g== X-Received: by 2002:a17:906:43c2:: with SMTP id j2mr14831413ejn.136.1555110280996; Fri, 12 Apr 2019 16:04:40 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id p23sm6831869ejr.74.2019.04.12.16.04.40 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 12 Apr 2019 16:04:40 -0700 (PDT) Date: Fri, 12 Apr 2019 16:04:40 -0700 (PDT) X-Google-Original-Date: Fri, 12 Apr 2019 23:04:38 GMT Message-Id: <44d5371ae6808ec40e8f52c3dc258a85c878b27e.1555110278.git.gitgitgadget@gmail.com> In-Reply-To: References: From: "Rohit Ashiwal via GitGitGadget" Subject: [PATCH 2/2] archive: avoid spawning `gzip` Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: Junio C Hamano , Rohit Ashiwal Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Rohit Ashiwal As we already link to the zlib library, we can perform the compression without even requiring gzip on the host machine. Signed-off-by: Rohit Ashiwal Signed-off-by: Johannes Schindelin Signed-off-by: Rene Scharfe --- archive-tar.c | 34 +++++++++++++++++++++++++--------- 1 file changed, 25 insertions(+), 9 deletions(-) diff --git a/archive-tar.c b/archive-tar.c index ba37dad27c..5979ed14b7 100644 --- a/archive-tar.c +++ b/archive-tar.c @@ -466,18 +466,34 @@ static int write_tar_filter_archive(const struct archiver *ar, filter.use_shell = 1; filter.in = -1; - if (start_command(&filter) < 0) - die_errno(_("unable to start '%s' filter"), argv[0]); - close(1); - if (dup2(filter.in, 1) < 0) - die_errno(_("unable to redirect descriptor")); - close(filter.in); + if (!strcmp("gzip -cn", ar->data)) { + char outmode[4] = "wb\0"; + + if (args->compression_level >= 0 && args->compression_level <= 9) + outmode[2] = '0' + args->compression_level; + + gzip = gzdopen(fileno(stdout), outmode); + if (!gzip) + die(_("Could not gzdopen stdout")); + } else { + if (start_command(&filter) < 0) + die_errno(_("unable to start '%s' filter"), argv[0]); + close(1); + if (dup2(filter.in, 1) < 0) + die_errno(_("unable to redirect descriptor")); + close(filter.in); + } r = write_tar_archive(ar, args); - close(1); - if (finish_command(&filter) != 0) - die(_("'%s' filter reported error"), argv[0]); + if (gzip) { + if (gzclose(gzip) != Z_OK) + die(_("gzclose failed")); + } else { + close(1); + if (finish_command(&filter) != 0) + die(_("'%s' filter reported error"), argv[0]); + } strbuf_release(&cmd); return r;