From patchwork Mon Sep 30 01:50:46 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matheus Tavares X-Patchwork-Id: 11165907 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id EB2E51599 for ; Mon, 30 Sep 2019 01:51:17 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id BE5852082F for ; Mon, 30 Sep 2019 01:51:17 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=usp-br.20150623.gappssmtp.com header.i=@usp-br.20150623.gappssmtp.com header.b="XpvuILXF" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729188AbfI3BvK (ORCPT ); Sun, 29 Sep 2019 21:51:10 -0400 Received: from mail-qk1-f194.google.com ([209.85.222.194]:46992 "EHLO mail-qk1-f194.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726360AbfI3BvK (ORCPT ); Sun, 29 Sep 2019 21:51:10 -0400 Received: by mail-qk1-f194.google.com with SMTP id 201so6389458qkd.13 for ; Sun, 29 Sep 2019 18:51:09 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=usp-br.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=k+59ScLgg6gvCPiPaVrgq45X0/YVz/3sa5rzKCPOuJo=; b=XpvuILXFfILWfPlwk1oV79SuUaT7b+fj007zNZFSHE7M+5xFMOXyBi7PDGhyNk0xkV EuGKHBWCTmZZ3VrNFZ5AXuDpwa6TVqHMqqyo0fvJgt05HOP9XWm+uiMQdY7Hsu/QxW7u aprlrdTxqsmS3SZeFwxi6r5YbV/TM/5a/TlN838LV9q0Ra6IgAB44qACQhmTZ8s3aSZm h56lKwuwhHKCUmy1r9EBNOSRDxm0tWFAegsqXVkGaggpFYa0JR7QxIm3EvBZzhNmMzp1 QQYAHHwYk+TWDLA0IZInYcrtGjfDuRvnMihEpR8dv5jx2/7ZBrGVm9ZVXHoke0hScVDa 1dTA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=k+59ScLgg6gvCPiPaVrgq45X0/YVz/3sa5rzKCPOuJo=; b=r7TyE97TRAcfmcmDgevcv+sc5FpzCdxOzfdZUTzaz3Q4d2H76fEbGivHX9/TlFntmx ItiFtsXGnn5s9M0xpak72CD0MIs8RQ34dtW3q5gCcQy25ANMXPPDZJ4ww+hWUp0m0CPm C5tRhtdq8AzHNkGMkMh9UzL6tROeuXeaQUYqbTOpRo4jRPGOedpDOz2hGufe3losIiL2 J5n9mYgmK2ZiNYPVidjyETLXzwL9nIfvtJU1z60myIgLojGKw+gWyHW09/S63FPC4DJD OX+VZt0DfLBAij+l6i/JLswiLircOlmPszx4ovT1Vb/pe+ZeCdj8SIsneYkFk1yoHoJh wyUA== X-Gm-Message-State: APjAAAVbmEd3eLkgnySx62/NCdPNalJd7neeXU26Xqr+aijZofu85ooQ H+rSWwCZsN91Q7+HSbY1zWfd+EsvXKI= X-Google-Smtp-Source: APXvYqze5zsm6dgRkWAvWb5E6TeyXGgQgK6u2LaUk2P1+d5gRVVHuP9v6pWAfOPtwhhcnSZWDbZsWw== X-Received: by 2002:a05:620a:119a:: with SMTP id b26mr16577949qkk.39.1569808268889; Sun, 29 Sep 2019 18:51:08 -0700 (PDT) Received: from mango.spo.virtua.com.br ([2804:14c:81:942d::3]) by smtp.gmail.com with ESMTPSA id f11sm4706954qkk.76.2019.09.29.18.51.06 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 29 Sep 2019 18:51:08 -0700 (PDT) From: Matheus Tavares To: git@vger.kernel.org Cc: christian.couder@gmail.com, olyatelezhnaya@gmail.com, pclouds@gmail.com, gitster@pobox.com, jrnieder@gmail.com Subject: [PATCH v2 00/11] grep: improve threading and fix race conditions Date: Sun, 29 Sep 2019 22:50:46 -0300 Message-Id: X-Mailer: git-send-email 2.23.0 In-Reply-To: References: MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org This series focus on re-enabling threads at git-grep for the non-worktree case. They are currently disabled due to being slower than single-threaded grep in this case. However, by allowing parallel zlib inflation when reading objects, speedups of up to 3x were observed. The patchset also contains some fixes for race conditions found in the worktree git-grep and thread optimizations to hopefully increase overall performance. This version was almost entirely re-written from scratch so I thought a range-diff wouldn't be very useful. The major differences from the first one are the race condition fixes and being able to run --textconv and --recurse-submodules threaded now. Matheus Tavares (11): grep: fix race conditions on userdiff calls grep: fix race conditions at grep_submodule() grep: fix racy calls in grep_objects() replace-object: make replace operations thread-safe object-store: allow threaded access to object reading grep: replace grep_read_mutex by internal obj read lock submodule-config: add skip_if_read option to repo_read_gitmodules() grep: allow submodule functions to run in parallel grep: protect packed_git [re-]initialization grep: re-enable threads in non-worktree case grep: move driver pre-load out of critical section .tsan-suppressions | 6 +++ Documentation/git-grep.txt | 11 +++++ builtin/grep.c | 90 +++++++++++++++++++------------------- grep.c | 32 ++++++++------ grep.h | 13 ------ object-store.h | 37 ++++++++++++++++ object.c | 2 + packfile.c | 9 ++++ replace-object.c | 11 ++++- replace-object.h | 7 ++- sha1-file.c | 57 +++++++++++++++++++++--- submodule-config.c | 18 +++----- submodule-config.h | 2 +- unpack-trees.c | 4 +- 14 files changed, 205 insertions(+), 94 deletions(-)