From patchwork Wed Feb 26 20:09:33 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Linus Arver via GitGitGadget X-Patchwork-Id: 11407127 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id ED09792A for ; Wed, 26 Feb 2020 20:09:42 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id B8FB224656 for ; Wed, 26 Feb 2020 20:09:42 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="GJ2fX0Hs" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727443AbgBZUJl (ORCPT ); Wed, 26 Feb 2020 15:09:41 -0500 Received: from mail-ed1-f65.google.com ([209.85.208.65]:37066 "EHLO mail-ed1-f65.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727260AbgBZUJk (ORCPT ); Wed, 26 Feb 2020 15:09:40 -0500 Received: by mail-ed1-f65.google.com with SMTP id t7so343629edr.4 for ; Wed, 26 Feb 2020 12:09:38 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=CYgdvY/jwh6P+Sat0uPvvb/t1OcImnRTUZNc+CLAzmY=; b=GJ2fX0HsEzfGlXDjWr6EgGNLiqvDhVeD37mE+e73zUtuC1FuYikJo+gdNh5S895juh O9ttL5M8iaU4LPWLSKZwj62cZ7dUvoTkX7dOkUJ+OS9I41fHVGX5qo0T25vtRsWiRs2e LCBc8elsldNmKZ0FrFF1xGqm14dK1p3/VMOm/lN3Bxr81Lv8ENMBzAKqrZMv32Y9SlsG nYjY060m93ARCxMsd5Vc+68/GX624cbHkfaWdsr3mbLuH9V6+AJegnEROgnJLWLX2uOt 4E0k52EJBEFBG3pSSz7w8cNh8+lrTzzq638pTG/X0MecS4K2lWVViFfiAG035u9x3o3P 23cQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=CYgdvY/jwh6P+Sat0uPvvb/t1OcImnRTUZNc+CLAzmY=; b=SZneNwQMJsuV+5rz2gz7+uJNdjm6JPvtRvIi187bEEuG1mLhIOAurgmE9BOO+ouL5C rZVfOYeSbkbfglaUPfMjf2Zc6o5hCZxJt0b9btJrPPlPkc4cpl9xFimhgStlneC8FRRV eorCFyOXSoR+5Po0GaEh5anOuGkmJCnbAfBe1rUqyqXR9z10YTDYAPtT0UsRGnH/CqT6 fTHwXujix8N8GGMOa9mlAWTDi8R5CcCV9fU6vH3+m9GEhLVQl066mhDczrF4Mmec2P85 XdjFqwSBgVPC3lXHp/u787sLHyKqeZkhdxTFD95hkfrk7HxTQbOwsd88X/mBUcwjGJjo 4zzA== X-Gm-Message-State: APjAAAXDc5yMtjIfz9ReEIRmB9k1c137YE2c7fPlV1JVpGzTVTaGI03K +D3MdgruIX5HtSJ0Xq7Y+M0nYHHN X-Google-Smtp-Source: APXvYqxYjsBFUHMrPyAOEwIGswypX2QfxeGhuS7uBP+vSQE3ya798UIr8cpBaUxei3u5dr/SXVrrCA== X-Received: by 2002:a17:906:a81:: with SMTP id y1mr455134ejf.270.1582747777534; Wed, 26 Feb 2020 12:09:37 -0800 (PST) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id f10sm202135eds.31.2020.02.26.12.09.37 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 26 Feb 2020 12:09:37 -0800 (PST) Message-Id: <9900cacbfefb46610114702e6d48d8020f3f2866.1582747775.git.gitgitgadget@gmail.com> In-Reply-To: References: From: "Johannes Schindelin via GitGitGadget" Date: Wed, 26 Feb 2020 20:09:33 +0000 Subject: [PATCH 1/3] t/lib-httpd: avoid using BSD's sed Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: Johannes Schindelin , Johannes Schindelin Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Johannes Schindelin Among other differences relative to GNU sed, BSD sed always ends its output with a trailing newline, even if the input did not have such a trailing newline. Surprisingly, this makes three httpd-based tests fail on macOS: t5616, t5702 and t5703. ("Surprisingly" because those tests have been around for some time, but apparently nobody runs them on macOS with a working Apache2 setup.) The reason is that we use `sed` in those tests to filter the response of the web server. Apart from the fact that we use GNU constructs (such as using a space after the `c` command instead of a backslash and a newline), we have another problem: BSD sed LF-only newlines while webservers are supposed to use CR/LF ones. Even worse, t5616 uses `sed` to replace a binary part of the response with a new binary part (kind of hoping that the replaced binary part does not contain a 0x0a byte which would be interpreted as a newline). To that end, it calls on Perl to read the binary pack file and hex-encode it, then calls on `sed` to prefix every hex digit pair with a `\x` in order to construct the text that the `c` statement of the `sed` invocation is supposed to insert. So we call Perl and sed to construct a sed statement. The final nail in the coffin is that BSD sed does not even interpret those `\x` constructs. Let's just replace all of that by Perl snippets. With Perl, at least, we do not have to deal with GNU vs BSD semantics, we do not have to worry about unwanted trailing newlines, and we do not have to spawn commands to construct arguments for other commands to be spawned (i.e. we can avoid a whole lot of shell scripting complexity). The upshot is that this fixes t5616, t5702 and t5703 on macOS with Apache2. Signed-off-by: Johannes Schindelin --- t/lib-httpd.sh | 2 +- t/lib-httpd/apache.conf | 6 ++--- t/lib-httpd/apply-one-time-perl.sh | 27 +++++++++++++++++++++ t/lib-httpd/apply-one-time-sed.sh | 24 ------------------- t/t5537-fetch-shallow.sh | 10 ++++---- t/t5616-partial-clone.sh | 38 +++++++++++++++++------------- t/t5702-protocol-v2.sh | 12 +++++----- t/t5703-upload-pack-ref-in-want.sh | 6 ++--- 8 files changed, 66 insertions(+), 59 deletions(-) create mode 100644 t/lib-httpd/apply-one-time-perl.sh delete mode 100644 t/lib-httpd/apply-one-time-sed.sh diff --git a/t/lib-httpd.sh b/t/lib-httpd.sh index 656997b4d66..1449ee95e9e 100644 --- a/t/lib-httpd.sh +++ b/t/lib-httpd.sh @@ -132,7 +132,7 @@ prepare_httpd() { install_script broken-smart-http.sh install_script error-smart-http.sh install_script error.sh - install_script apply-one-time-sed.sh + install_script apply-one-time-perl.sh ln -s "$LIB_HTTPD_MODULE_PATH" "$HTTPD_ROOT_PATH/modules" diff --git a/t/lib-httpd/apache.conf b/t/lib-httpd/apache.conf index 5c1c86c193a..994e5290d63 100644 --- a/t/lib-httpd/apache.conf +++ b/t/lib-httpd/apache.conf @@ -113,7 +113,7 @@ Alias /auth/dumb/ www/auth/dumb/ SetEnv GIT_EXEC_PATH ${GIT_EXEC_PATH} SetEnv GIT_HTTP_EXPORT_ALL - + SetEnv GIT_EXEC_PATH ${GIT_EXEC_PATH} SetEnv GIT_HTTP_EXPORT_ALL @@ -122,7 +122,7 @@ ScriptAliasMatch /smart_*[^/]*/(.*) ${GIT_EXEC_PATH}/git-http-backend/$1 ScriptAlias /broken_smart/ broken-smart-http.sh/ ScriptAlias /error_smart/ error-smart-http.sh/ ScriptAlias /error/ error.sh/ -ScriptAliasMatch /one_time_sed/(.*) apply-one-time-sed.sh/$1 +ScriptAliasMatch /one_time_perl/(.*) apply-one-time-perl.sh/$1 Options FollowSymlinks @@ -135,7 +135,7 @@ ScriptAliasMatch /one_time_sed/(.*) apply-one-time-sed.sh/$1 Options ExecCGI - + Options ExecCGI diff --git a/t/lib-httpd/apply-one-time-perl.sh b/t/lib-httpd/apply-one-time-perl.sh new file mode 100644 index 00000000000..09a0abdff7c --- /dev/null +++ b/t/lib-httpd/apply-one-time-perl.sh @@ -0,0 +1,27 @@ +#!/bin/sh + +# If "one-time-perl" exists in $HTTPD_ROOT_PATH, run perl on the HTTP response, +# using the contents of "one-time-perl" as the perl command to be run. If the +# response was modified as a result, delete "one-time-perl" so that subsequent +# HTTP responses are no longer modified. +# +# This can be used to simulate the effects of the repository changing in +# between HTTP request-response pairs. +if test -f one-time-perl +then + LC_ALL=C + export LC_ALL + + "$GIT_EXEC_PATH/git-http-backend" >out + perl -pe "$(cat one-time-perl)" out >out_modified + + if cmp -s out out_modified + then + cat out + else + cat out_modified + rm one-time-perl + fi +else + "$GIT_EXEC_PATH/git-http-backend" +fi diff --git a/t/lib-httpd/apply-one-time-sed.sh b/t/lib-httpd/apply-one-time-sed.sh deleted file mode 100644 index bf7689d0202..00000000000 --- a/t/lib-httpd/apply-one-time-sed.sh +++ /dev/null @@ -1,24 +0,0 @@ -#!/bin/sh - -# If "one-time-sed" exists in $HTTPD_ROOT_PATH, run sed on the HTTP response, -# using the contents of "one-time-sed" as the sed command to be run. If the -# response was modified as a result, delete "one-time-sed" so that subsequent -# HTTP responses are no longer modified. -# -# This can be used to simulate the effects of the repository changing in -# between HTTP request-response pairs. -if test -f one-time-sed -then - "$GIT_EXEC_PATH/git-http-backend" >out - sed "$(cat one-time-sed)" out >out_modified - - if cmp -s out out_modified - then - cat out - else - cat out_modified - rm one-time-sed - fi -else - "$GIT_EXEC_PATH/git-http-backend" -fi diff --git a/t/t5537-fetch-shallow.sh b/t/t5537-fetch-shallow.sh index 9e16512fe31..4f10057e9f1 100755 --- a/t/t5537-fetch-shallow.sh +++ b/t/t5537-fetch-shallow.sh @@ -237,7 +237,7 @@ test_expect_success 'shallow fetches check connectivity before writing shallow f git -C "$REPO" config protocol.version 2 && git -C client config protocol.version 2 && - git -C client fetch --depth=2 "$HTTPD_URL/one_time_sed/repo" master:a_branch && + git -C client fetch --depth=2 "$HTTPD_URL/one_time_perl/repo" master:a_branch && # Craft a situation in which the server sends back an unshallow request # with an empty packfile. This is done by refetching with a shorter @@ -246,13 +246,13 @@ test_expect_success 'shallow fetches check connectivity before writing shallow f printf "$(test_oid sed)" \ "$(git -C "$REPO" rev-parse HEAD)" \ "$(git -C "$REPO" rev-parse HEAD^)" \ - >"$HTTPD_ROOT_PATH/one-time-sed" && + >"$HTTPD_ROOT_PATH/one-time-perl" && test_must_fail env GIT_TEST_SIDEBAND_ALL=0 git -C client \ - fetch --depth=1 "$HTTPD_URL/one_time_sed/repo" \ + fetch --depth=1 "$HTTPD_URL/one_time_perl/repo" \ master:a_branch && - # Ensure that the one-time-sed script was used. - ! test -e "$HTTPD_ROOT_PATH/one-time-sed" && + # Ensure that the one-time-perl script was used. + ! test -e "$HTTPD_ROOT_PATH/one-time-perl" && # Ensure that the resulting repo is consistent, despite our failure to # fetch. diff --git a/t/t5616-partial-clone.sh b/t/t5616-partial-clone.sh index 9a9178fd281..0eb5b1c47b1 100755 --- a/t/t5616-partial-clone.sh +++ b/t/t5616-partial-clone.sh @@ -398,14 +398,18 @@ intersperse () { sed 's/\(..\)/'$1'\1/g' } -# Create a one-time-sed command to replace the existing packfile with $1. +# Create a one-time-perl command to replace the existing packfile with $1. replace_packfile () { # The protocol requires that the packfile be sent in sideband 1, hence # the extra \x01 byte at the beginning. - printf "1,/packfile/!c %04x\\\\x01%s0000" \ - "$(($(wc -c <$1) + 5))" \ - "$(hex_unpack <$1 | intersperse '\\x')" \ - >"$HTTPD_ROOT_PATH/one-time-sed" + cp $1 "$HTTPD_ROOT_PATH/one-time-pack" && + echo 'if (/packfile/) { + print; + my $length = -s "one-time-pack"; + printf "%04x\x01", $length + 5; + print `cat one-time-pack` . "0000"; + last + }' >"$HTTPD_ROOT_PATH/one-time-perl" } test_expect_success 'upon cloning, check that all refs point to objects' ' @@ -429,16 +433,16 @@ test_expect_success 'upon cloning, check that all refs point to objects' ' # \x01 byte at the beginning. replace_packfile incomplete.pack && - # Use protocol v2 because the sed command looks for the "packfile" + # Use protocol v2 because the perl command looks for the "packfile" # section header. test_config -C "$SERVER" protocol.version 2 && test_must_fail git -c protocol.version=2 clone \ - --filter=blob:none $HTTPD_URL/one_time_sed/server repo 2>err && + --filter=blob:none $HTTPD_URL/one_time_perl/server repo 2>err && test_i18ngrep "did not send all necessary objects" err && - # Ensure that the one-time-sed script was used. - ! test -e "$HTTPD_ROOT_PATH/one-time-sed" + # Ensure that the one-time-perl script was used. + ! test -e "$HTTPD_ROOT_PATH/one-time-perl" ' test_expect_success 'when partial cloning, tolerate server not sending target of tag' ' @@ -469,17 +473,17 @@ test_expect_success 'when partial cloning, tolerate server not sending target of # \x01 byte at the beginning. replace_packfile incomplete.pack && - # Use protocol v2 because the sed command looks for the "packfile" + # Use protocol v2 because the perl command looks for the "packfile" # section header. test_config -C "$SERVER" protocol.version 2 && # Exercise to make sure it works. git -c protocol.version=2 clone \ - --filter=blob:none $HTTPD_URL/one_time_sed/server repo 2> err && + --filter=blob:none $HTTPD_URL/one_time_perl/server repo 2> err && ! grep "missing object referenced by" err && - # Ensure that the one-time-sed script was used. - ! test -e "$HTTPD_ROOT_PATH/one-time-sed" + # Ensure that the one-time-perl script was used. + ! test -e "$HTTPD_ROOT_PATH/one-time-perl" ' test_expect_success 'tolerate server sending REF_DELTA against missing promisor objects' ' @@ -502,7 +506,7 @@ test_expect_success 'tolerate server sending REF_DELTA against missing promisor # Clone. The client has deltabase_have but not deltabase_missing. git -c protocol.version=2 clone --no-checkout \ - --filter=blob:none $HTTPD_URL/one_time_sed/server repo && + --filter=blob:none $HTTPD_URL/one_time_perl/server repo && git -C repo hash-object -w -- "$SERVER/have.txt" && # Sanity check to ensure that the client does not have @@ -543,7 +547,7 @@ test_expect_success 'tolerate server sending REF_DELTA against missing promisor replace_packfile thin.pack && - # Use protocol v2 because the sed command looks for the "packfile" + # Use protocol v2 because the perl command looks for the "packfile" # section header. test_config -C "$SERVER" protocol.version 2 && @@ -556,8 +560,8 @@ test_expect_success 'tolerate server sending REF_DELTA against missing promisor grep "want $(cat deltabase_missing)" trace && ! grep "want $(cat deltabase_have)" trace && - # Ensure that the one-time-sed script was used. - ! test -e "$HTTPD_ROOT_PATH/one-time-sed" + # Ensure that the one-time-perl script was used. + ! test -e "$HTTPD_ROOT_PATH/one-time-perl" ' # DO NOT add non-httpd-specific tests here, because the last part of this diff --git a/t/t5702-protocol-v2.sh b/t/t5702-protocol-v2.sh index 7fd7102c874..5039e66dc47 100755 --- a/t/t5702-protocol-v2.sh +++ b/t/t5702-protocol-v2.sh @@ -712,11 +712,11 @@ test_expect_success 'when server sends "ready", expect DELIM' ' # After "ready" in the acknowledgments section, pretend that a FLUSH # (0000) was sent instead of a DELIM (0001). - printf "/ready/,$ s/0001/0000/" \ - >"$HTTPD_ROOT_PATH/one-time-sed" && + printf "\$ready = 1 if /ready/; \$ready && s/0001/0000/" \ + >"$HTTPD_ROOT_PATH/one-time-perl" && test_must_fail git -C http_child -c protocol.version=2 \ - fetch "$HTTPD_URL/one_time_sed/http_parent" 2> err && + fetch "$HTTPD_URL/one_time_perl/http_parent" 2> err && test_i18ngrep "expected packfile to be sent after .ready." err ' @@ -737,12 +737,12 @@ test_expect_success 'when server does not send "ready", expect FLUSH' ' # After the acknowledgments section, pretend that a DELIM # (0001) was sent instead of a FLUSH (0000). - printf "/acknowledgments/,$ s/0000/0001/" \ - >"$HTTPD_ROOT_PATH/one-time-sed" && + printf "\$ack = 1 if /acknowledgments/; \$ack && s/0000/0001/" \ + >"$HTTPD_ROOT_PATH/one-time-perl" && test_must_fail env GIT_TRACE_PACKET="$(pwd)/log" git -C http_child \ -c protocol.version=2 \ - fetch "$HTTPD_URL/one_time_sed/http_parent" 2> err && + fetch "$HTTPD_URL/one_time_perl/http_parent" 2> err && grep "fetch< .*acknowledgments" log && ! grep "fetch< .*ready" log && test_i18ngrep "expected no other sections to be sent after no .ready." err diff --git a/t/t5703-upload-pack-ref-in-want.sh b/t/t5703-upload-pack-ref-in-want.sh index 8aeeaac5091..7fba3063bf9 100755 --- a/t/t5703-upload-pack-ref-in-want.sh +++ b/t/t5703-upload-pack-ref-in-want.sh @@ -314,7 +314,7 @@ test_expect_success 'setup repos for change-while-negotiating test' ' test_commit m3 && git tag -d m2 m3 ) && - git -C "$LOCAL_PRISTINE" remote set-url origin "http://127.0.0.1:$LIB_HTTPD_PORT/one_time_sed/repo" && + git -C "$LOCAL_PRISTINE" remote set-url origin "http://127.0.0.1:$LIB_HTTPD_PORT/one_time_perl/repo" && git -C "$LOCAL_PRISTINE" config protocol.version 2 ' @@ -327,7 +327,7 @@ inconsistency () { # RPCs during a single negotiation. oid1=$(git -C "$REPO" rev-parse $1) && oid2=$(git -C "$REPO" rev-parse $2) && - echo "s/$oid1/$oid2/" >"$HTTPD_ROOT_PATH/one-time-sed" + echo "s/$oid1/$oid2/" >"$HTTPD_ROOT_PATH/one-time-perl" } test_expect_success 'server is initially ahead - no ref in want' ' @@ -379,7 +379,7 @@ test_expect_success 'server loses a ref - ref in want' ' git -C "$REPO" config uploadpack.allowRefInWant true && rm -rf local && cp -r "$LOCAL_PRISTINE" local && - echo "s/master/raster/" >"$HTTPD_ROOT_PATH/one-time-sed" && + echo "s/master/raster/" >"$HTTPD_ROOT_PATH/one-time-perl" && test_must_fail git -C local fetch 2>err && test_i18ngrep "fatal: remote error: unknown ref refs/heads/raster" err From patchwork Wed Feb 26 20:09:34 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Linus Arver via GitGitGadget X-Patchwork-Id: 11407129 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id D2F6192A for ; Wed, 26 Feb 2020 20:09:43 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id B2B9724670 for ; Wed, 26 Feb 2020 20:09:43 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="bMw4FyaY" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727405AbgBZUJl (ORCPT ); Wed, 26 Feb 2020 15:09:41 -0500 Received: from mail-ed1-f67.google.com ([209.85.208.67]:33307 "EHLO mail-ed1-f67.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727335AbgBZUJk (ORCPT ); Wed, 26 Feb 2020 15:09:40 -0500 Received: by mail-ed1-f67.google.com with SMTP id r21so378446edq.0 for ; Wed, 26 Feb 2020 12:09:38 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=+ALkhmx9PbPWta5lAxlkaCU266Jj0kFtCZlFEDIWAoo=; b=bMw4FyaYOSKz1NqGpRBQncRvfYT9Hlm/+/2oV2UY0ohnad8FeM2/ZcpwsNpHUJ0iVB id5v/1dptFl+RfWYgcfWMfXzRDs9MFvDoBnZpRbjJm+WCRxxii9V8yWcsUFTAhgG+MoG cVz3Bj8QQTXuO5WtMlovsiGz64f1IwTkzsYC8VzBS33vDXXQlMbihN89A8dvGdZsFMMX r2fSNrjpoVTiLHkR+olIfaFKyN73H+Gk/6XZZEzEvjz4gt7v3Mn0adc8dGenApQdAyX2 Uw/MfWSNtYHbRlq+7E/owOnKFQEQTMpDmeZO0VZ2mdL0W0I5tGA3LyJdxJ3aEECDEkOE 6mCg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=+ALkhmx9PbPWta5lAxlkaCU266Jj0kFtCZlFEDIWAoo=; b=MMHZnarqrekbeuJOvbht1piizQee9TXljWbfI/kO9HWHuPVgYiG5+SfDbEg3C+66HN qjRQoMMUUs+42K9uoU7NHrDoY9TiuwurYxM1+oNY3dJC5URf4sFftFZaBPKIJx1q6Ueq 0LnI/kQcEuwPCdCtCsiaxqP9yU/VWoEggazAyVFY5W/y4O9+kfb/1bNEbanS3cc98FSF sWoVlyowYU299aQrd0s9Gkaihq1UGP8p9N3qAuf8MIROgqqc5c71pgXedrHxlKuosJkP cBSGZNW1Tp0+XPRAkUQ6geSuUBVSLK8MFqCGRIwLT0ynnzrpFNdkAeJWQlCEeVQ+kZ38 M7lQ== X-Gm-Message-State: APjAAAWFNBRppotxc4NkOXSImYiWMIlej0JDZr+Aq4pbrHfhla+v4Fw5 QpuTE7zib+JasYI2vmjUsqAUAoKD X-Google-Smtp-Source: APXvYqwA7jyGpMFr7aSE/u989DL9loPzlJ2Cx23YK73WM2heEBCesvD+gK4tZNCp1rRc0CrSuMEXcg== X-Received: by 2002:aa7:ce13:: with SMTP id d19mr117430edv.296.1582747778173; Wed, 26 Feb 2020 12:09:38 -0800 (PST) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id bs4sm203960edb.16.2020.02.26.12.09.37 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 26 Feb 2020 12:09:37 -0800 (PST) Message-Id: <8739f8bac0f1c0a97ac16255807514bf7eba0020.1582747775.git.gitgitgadget@gmail.com> In-Reply-To: References: From: "Johannes Schindelin via GitGitGadget" Date: Wed, 26 Feb 2020 20:09:34 +0000 Subject: [PATCH 2/3] ci: prevent `perforce` from being quarantined Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: Johannes Schindelin , Johannes Schindelin Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Johannes Schindelin The most recent Azure Pipelines macOS agents enable what Apple calls "System Integrity Protection". This makes `p4d -V` hang: there is some sort of GUI dialog waiting for the user to acknowledge that the copied binaries are legit and may be executed, but on build agents, there is no user who could acknowledge that. Let's ask Homebrew specifically to _not_ quarantine the Perforce binaries. Helped-by: Aleksandr Chebotov Signed-off-by: Johannes Schindelin --- ci/install-dependencies.sh | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/ci/install-dependencies.sh b/ci/install-dependencies.sh index cd59855d73d..497fd32ca83 100755 --- a/ci/install-dependencies.sh +++ b/ci/install-dependencies.sh @@ -40,11 +40,11 @@ osx-clang|osx-gcc) test -z "$BREW_INSTALL_PACKAGES" || brew install $BREW_INSTALL_PACKAGES brew link --force gettext - brew cask install perforce || { + brew cask install --no-quarantine perforce || { # Update the definitions and try again cask_repo="$(brew --repository)"/Library/Taps/homebrew/homebrew-cask && git -C "$cask_repo" pull --no-stat && - brew cask install perforce + brew cask install --no-quarantine perforce } || brew install caskroom/cask/perforce case "$jobname" in From patchwork Wed Feb 26 20:09:35 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Linus Arver via GitGitGadget X-Patchwork-Id: 11407125 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 7F19614B4 for ; Wed, 26 Feb 2020 20:09:42 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 562A124670 for ; Wed, 26 Feb 2020 20:09:42 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="NwwYnMzW" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727450AbgBZUJl (ORCPT ); Wed, 26 Feb 2020 15:09:41 -0500 Received: from mail-ed1-f68.google.com ([209.85.208.68]:35268 "EHLO mail-ed1-f68.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727277AbgBZUJl (ORCPT ); Wed, 26 Feb 2020 15:09:41 -0500 Received: by mail-ed1-f68.google.com with SMTP id c7so361049edu.2 for ; Wed, 26 Feb 2020 12:09:40 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=PvLcEFm2U2RuVDEvvMsqbjR3vL2S4N4DlN6D8Aeo/gU=; b=NwwYnMzWBotsM8D1lfZoC3Di3EsPyDRCJjqmTMuvjcsCqz3A0Ej3omqDyx6eR7aVF4 tDBkRjBln2Q9mtIx9o+I2XlI18jsV0AZVCTWoGfUrAQ3bypR3tgwkYzbPplS7BMErYhh BKW8aFwWHIIif9BXj5YwCZ6IVIaT+ZgZrrkQqsEp24DxOiGt4eRUyDSEKtNgle79wRqL UYKv9B7pCUxX+U03ac8TD5msOCmu/TWmSQ+WSHAfNeREwQVnxN4D1Dy+U5bfgBx+yV+F 4tgVvC7OF6NVejJWnFu9dZD1+jW2+vzQWjse+2Fb+dEEtlq8Q8tHxnLHUgdv2yFCqySc obHQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=PvLcEFm2U2RuVDEvvMsqbjR3vL2S4N4DlN6D8Aeo/gU=; b=cZOcNYtmGUYCtyaINCcUqJVN+qMIlXGkwMUjyxLpxFyLGgrVloyzcIkXDoUr7RUIBU r+6X+niPxdlSHy0aS0M58buTUAlci0VcpZlA8ekEscOAXgmMFDZNBp9ifLFFexWY8GEL fMGoaJk6pANkfprua+Nez6EbMA6JCVNXtGVW9KOSFzkBSAj5VwpGlBxljBfjrnOcNYQX dXfN0HpdAdMlKHBU1a6G9XNj7f3OxOlLFb58zwI6NMxna06wliqfHpaJ4ygfWJSmcDpb J/nw44271tsOBSjyEeS0EzMbrUCi6wqSCYCZiS6e2aBkH+yB0aL+G4pq4j9TKQdSMcH9 qAnA== X-Gm-Message-State: APjAAAUtyOTKYpG0Myyfe7cuQGPoEwnMOqldPxoeaOpbuDKzTqNsetG6 AlkyLTWk5L6i7G8Cg6RJWGMZprv2 X-Google-Smtp-Source: APXvYqxoqwAthh6z2VQugR5GrjRba9gHoR2V2g/AhIS9V6DKEn7B0K6lg05BXJUJdaGSPlYKJmBONQ== X-Received: by 2002:aa7:c946:: with SMTP id h6mr132219edt.186.1582747778904; Wed, 26 Feb 2020 12:09:38 -0800 (PST) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id t11sm203771edj.78.2020.02.26.12.09.38 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 26 Feb 2020 12:09:38 -0800 (PST) Message-Id: In-Reply-To: References: From: "Johannes Schindelin via GitGitGadget" Date: Wed, 26 Feb 2020 20:09:35 +0000 Subject: [PATCH 3/3] Azure Pipeline: switch to the latest agent pools Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: Johannes Schindelin , Johannes Schindelin Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Johannes Schindelin It would seem that at least the `vs2015-win2012r2` pool (which we use via its old name, `Hosted`) is about to be phased out. Let's switch before that. While at it, use the newer pool names as suggested at https://docs.microsoft.com/en-us/azure/devops/pipelines/agents/hosted?view=azure-devops#use-a-microsoft-hosted-agent Signed-off-by: Johannes Schindelin --- azure-pipelines.yml | 37 +++++++++++++++++++++++++------------ 1 file changed, 25 insertions(+), 12 deletions(-) diff --git a/azure-pipelines.yml b/azure-pipelines.yml index af2a5ea4845..675c3a43c9c 100644 --- a/azure-pipelines.yml +++ b/azure-pipelines.yml @@ -5,7 +5,8 @@ jobs: - job: windows_build displayName: Windows Build condition: succeeded() - pool: Hosted + pool: + vmImage: windows-latest timeoutInMinutes: 240 steps: - powershell: | @@ -61,7 +62,8 @@ jobs: displayName: Windows Test dependsOn: windows_build condition: succeeded() - pool: Hosted + pool: + vmImage: windows-latest timeoutInMinutes: 240 strategy: parallel: 10 @@ -133,7 +135,8 @@ jobs: - job: vs_build displayName: Visual Studio Build condition: succeeded() - pool: Hosted VS2017 + pool: + vmImage: windows-latest timeoutInMinutes: 240 steps: - powershell: | @@ -181,6 +184,7 @@ jobs: platform: x64 configuration: Release maximumCpuCount: 4 + msbuildArguments: /p:PlatformToolset=v142 - powershell: | & compat\vcbuild\vcpkg_copy_dlls.bat release if (!$?) { exit(1) } @@ -224,7 +228,8 @@ jobs: displayName: Visual Studio Test dependsOn: vs_build condition: succeeded() - pool: Hosted + pool: + vmImage: windows-latest timeoutInMinutes: 240 strategy: parallel: 10 @@ -292,7 +297,8 @@ jobs: - job: linux_clang displayName: linux-clang condition: succeeded() - pool: Hosted Ubuntu 1604 + pool: + vmImage: ubuntu-latest steps: - bash: | test "$GITFILESHAREPWD" = '$(gitfileshare.pwd)' || ci/mount-fileshare.sh //gitfileshare.file.core.windows.net/test-cache gitfileshare "$GITFILESHAREPWD" "$HOME/test-cache" || exit 1 @@ -330,7 +336,8 @@ jobs: - job: linux_gcc displayName: linux-gcc condition: succeeded() - pool: Hosted Ubuntu 1604 + pool: + vmImage: ubuntu-latest steps: - bash: | test "$GITFILESHAREPWD" = '$(gitfileshare.pwd)' || ci/mount-fileshare.sh //gitfileshare.file.core.windows.net/test-cache gitfileshare "$GITFILESHAREPWD" "$HOME/test-cache" || exit 1 @@ -367,7 +374,8 @@ jobs: - job: osx_clang displayName: osx-clang condition: succeeded() - pool: Hosted macOS + pool: + vmImage: macOS-latest steps: - bash: | test "$GITFILESHAREPWD" = '$(gitfileshare.pwd)' || ci/mount-fileshare.sh //gitfileshare.file.core.windows.net/test-cache gitfileshare "$GITFILESHAREPWD" "$HOME/test-cache" || exit 1 @@ -402,7 +410,8 @@ jobs: - job: osx_gcc displayName: osx-gcc condition: succeeded() - pool: Hosted macOS + pool: + vmImage: macOS-latest steps: - bash: | test "$GITFILESHAREPWD" = '$(gitfileshare.pwd)' || ci/mount-fileshare.sh //gitfileshare.file.core.windows.net/test-cache gitfileshare "$GITFILESHAREPWD" "$HOME/test-cache" || exit 1 @@ -435,7 +444,8 @@ jobs: - job: gettext_poison displayName: GETTEXT_POISON condition: succeeded() - pool: Hosted Ubuntu 1604 + pool: + vmImage: ubuntu-latest steps: - bash: | test "$GITFILESHAREPWD" = '$(gitfileshare.pwd)' || ci/mount-fileshare.sh //gitfileshare.file.core.windows.net/test-cache gitfileshare "$GITFILESHAREPWD" "$HOME/test-cache" || exit 1 @@ -472,7 +482,8 @@ jobs: - job: linux32 displayName: Linux32 condition: succeeded() - pool: Hosted Ubuntu 1604 + pool: + vmImage: ubuntu-latest steps: - bash: | test "$GITFILESHAREPWD" = '$(gitfileshare.pwd)' || ci/mount-fileshare.sh //gitfileshare.file.core.windows.net/test-cache gitfileshare "$GITFILESHAREPWD" "$HOME/test-cache" || exit 1 @@ -506,7 +517,8 @@ jobs: - job: static_analysis displayName: StaticAnalysis condition: succeeded() - pool: Hosted Ubuntu 1604 + pool: + vmImage: ubuntu-latest steps: - bash: | test "$GITFILESHAREPWD" = '$(gitfileshare.pwd)' || ci/mount-fileshare.sh //gitfileshare.file.core.windows.net/test-cache gitfileshare "$GITFILESHAREPWD" "$HOME/test-cache" || exit 1 @@ -526,7 +538,8 @@ jobs: - job: documentation displayName: Documentation condition: succeeded() - pool: Hosted Ubuntu 1604 + pool: + vmImage: ubuntu-latest steps: - bash: | test "$GITFILESHAREPWD" = '$(gitfileshare.pwd)' || ci/mount-fileshare.sh //gitfileshare.file.core.windows.net/test-cache gitfileshare "$GITFILESHAREPWD" "$HOME/test-cache" || exit 1