From patchwork Fri Jul 19 23:08:52 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matheus Tavares X-Patchwork-Id: 11050687 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id A444313A4 for ; Fri, 19 Jul 2019 23:09:22 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 9293C289CB for ; Fri, 19 Jul 2019 23:09:22 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 83D46289CF; Fri, 19 Jul 2019 23:09:22 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id DE1AC289CB for ; Fri, 19 Jul 2019 23:09:21 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1732550AbfGSXJU (ORCPT ); Fri, 19 Jul 2019 19:09:20 -0400 Received: from mail-qk1-f193.google.com ([209.85.222.193]:42531 "EHLO mail-qk1-f193.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728747AbfGSXJU (ORCPT ); Fri, 19 Jul 2019 19:09:20 -0400 Received: by mail-qk1-f193.google.com with SMTP id 201so24475620qkm.9 for ; Fri, 19 Jul 2019 16:09:19 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=usp-br.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=sK6oRj88RopfGSLmDBaFIhdoCi5ioBLr8L3lZ5OJ7mI=; b=Z/2BbhyRaCMqHZ46/AzmoMF0JcCukSEFNnuRaoF8L1Ubfy1dQAp2umSs77L/bmcoD5 NeCWR0a2IRMxoSalckIRrDRQMopPoFS2sOqsGLMjinP5UmpAANzF2xqRStO5ykK/Agjn tLqYf5RKl6ENibWlNPOGdFAC/NfPafEsLl1AlXR/f1krci54dlJGkJwDSmQk96vbAHfX kiQzEw+Yw3V+q00ZrJDc76RZz8FLLdj9nqw9weuTSFUTMchqvjDL4m5FAVsGX3utbw5H H23SHg0Moh9tpHq5pNBxblGHOWYFoFNRwAF4Hc880Rpkc9IZgV0w31Y7etAP3vRPhx16 1pKA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=sK6oRj88RopfGSLmDBaFIhdoCi5ioBLr8L3lZ5OJ7mI=; b=RJSxHxpp/zOCd7aZVjfoUlxiDfm1oHmxFzF0nWbhDrfPYgFO09f/B0q3k2HzkRlnV4 m07460p09ZH4CTGmkauk42NN2dBAhN7TLdZV+STcFkMkAxdbrX/8qqWvLbDkY6LJjo6X DWeLHkfYNvQVCe+WUxSkqc/DnqUn+aeAFJtbty5pQm6rOlmK5rhRjfggApgiFqyqsE95 LG2AbMPdwHbET8uaNOGRZisU2/8xysn3Mt94b1fg0/wCMo2WntyASXzpISvKytaG0DXV yCXVyWHYoe7LhTOr/TUlZAEgiRKs4aHWeMqRZamXFheEM6O6shL2yDHfhM3yFM7k0+Mr LaLA== X-Gm-Message-State: APjAAAVvNpPfCMk5r1E2euXPpMv8HwTXeSRSLHRMk3Tz6xhPeY+dqfDa JCkeQQN4Ry0L15PDmu/tB18PECxsW3Q= X-Google-Smtp-Source: APXvYqxMPAfgHpKH/3dpz2mv7KlpBlM434yua4crBlKoWsQCIVdm48pnkTMLo5IrMt1j4B18lQJYJw== X-Received: by 2002:a37:d245:: with SMTP id f66mr37252573qkj.59.1563577758778; Fri, 19 Jul 2019 16:09:18 -0700 (PDT) Received: from localhost.localdomain ([170.82.202.216]) by smtp.gmail.com with ESMTPSA id s127sm14469035qkd.107.2019.07.19.16.09.15 (version=TLS1_3 cipher=AEAD-AES256-GCM-SHA384 bits=256/256); Fri, 19 Jul 2019 16:09:18 -0700 (PDT) From: Matheus Tavares To: git@vger.kernel.org Cc: Christian Couder , Olga Telezhnaya , =?utf-8?b?Tmd1eeG7hW4gVGjDoWkg?= =?utf-8?b?Tmfhu41jIER1eQ==?= , kernel-usp@googlegroups.com, Stefan Beller , Jeff King , Denton Liu , "brian m. carlson" , Junio C Hamano Subject: [GSoC][RFC WIP PATCH 1/3] object-store: make read_object_file_extended() thread-safe Date: Fri, 19 Jul 2019 20:08:52 -0300 Message-Id: <1525a0b55c4dddef09c9ea8ed26def21a3e790aa.1563570204.git.matheus.bernardino@usp.br> X-Mailer: git-send-email 2.22.0 In-Reply-To: References: MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Allow read_object_file_extended() to be called by multiple threads protecting it with a lock. The lock usage can be toggled with enable_obj_read_lock() and disable_obj_read_lock(). Probably there are many spots in read_object_file_extended()'s call chain that could be executed unlocked (and thus, in parallel). But, just to make sure, let's protect everthing for now, and go refining the lock step-by-step in the future. The only (and perhaps most important) exception is git_inflate. Since it is already thread-safe and takes a significant amount of time, the lock is released when entering this function, so that it can be performed in parallel. This should already bring good performance, because of inflation's time cost. Note that add_delta_base_cache() was also modified to skip adding already present cache entries. This wouldn't happen in the past, but now it's possible as phase I and phase III of unpack_entry() may execute concurrently. Signed-off-by: Matheus Tavares --- object-store.h | 4 ++++ packfile.c | 7 +++++++ sha1-file.c | 56 +++++++++++++++++++++++++++++++++++++++++++++----- 3 files changed, 62 insertions(+), 5 deletions(-) diff --git a/object-store.h b/object-store.h index 49f56ab8d9..8330ff3988 100644 --- a/object-store.h +++ b/object-store.h @@ -157,6 +157,10 @@ const char *loose_object_path(struct repository *r, struct strbuf *buf, void *map_loose_object(struct repository *r, const struct object_id *oid, unsigned long *size); +void enable_obj_read_lock(void); +void disable_obj_read_lock(void); +void obj_read_lock(void); +void obj_read_unlock(void); void *read_object_file_extended(struct repository *r, const struct object_id *oid, enum object_type *type, diff --git a/packfile.c b/packfile.c index c0d83fdfed..a50be4e5e5 100644 --- a/packfile.c +++ b/packfile.c @@ -1087,7 +1087,9 @@ unsigned long get_size_from_delta(struct packed_git *p, do { in = use_pack(p, w_curs, curpos, &stream.avail_in); stream.next_in = in; + obj_read_unlock(); st = git_inflate(&stream, Z_FINISH); + obj_read_lock(); curpos += stream.next_in - in; } while ((st == Z_OK || st == Z_BUF_ERROR) && stream.total_out < sizeof(delta_head)); @@ -1440,6 +1442,9 @@ static void add_delta_base_cache(struct packed_git *p, off_t base_offset, struct delta_base_cache_entry *ent = xmalloc(sizeof(*ent)); struct list_head *lru, *tmp; + if (get_delta_base_cache_entry(p, base_offset)) + return; + delta_base_cached += base_size; list_for_each_safe(lru, tmp, &delta_base_cache_lru) { @@ -1569,7 +1574,9 @@ static void *unpack_compressed_entry(struct packed_git *p, do { in = use_pack(p, w_curs, curpos, &stream.avail_in); stream.next_in = in; + obj_read_unlock(); st = git_inflate(&stream, Z_FINISH); + obj_read_lock(); if (!stream.avail_out) break; /* the payload is larger than it should be */ curpos += stream.next_in - in; diff --git a/sha1-file.c b/sha1-file.c index 888b6024d5..37cde4a494 100644 --- a/sha1-file.c +++ b/sha1-file.c @@ -1463,16 +1463,49 @@ int pretend_object_file(void *buf, unsigned long len, enum object_type type, return 0; } +static pthread_mutex_t obj_read_mutex; +static int obj_read_use_lock = 0; + +void enable_obj_read_lock(void) +{ + if (obj_read_use_lock) + return; + + obj_read_use_lock = 1; + pthread_mutex_init(&obj_read_mutex, NULL); +} + +void disable_obj_read_lock(void) +{ + if (!obj_read_use_lock) + return; + + obj_read_use_lock = 0; + pthread_mutex_destroy(&obj_read_mutex); +} + +void obj_read_lock(void) +{ + if(obj_read_use_lock) + pthread_mutex_lock(&obj_read_mutex); +} + +void obj_read_unlock(void) +{ + if(obj_read_use_lock) + pthread_mutex_unlock(&obj_read_mutex); +} + /* * This function dies on corrupt objects; the callers who want to * deal with them should arrange to call read_object() and give error * messages themselves. */ -void *read_object_file_extended(struct repository *r, - const struct object_id *oid, - enum object_type *type, - unsigned long *size, - int lookup_replace) +static void *do_read_object_file_extended(struct repository *r, + const struct object_id *oid, + enum object_type *type, + unsigned long *size, + int lookup_replace) { void *data; const struct packed_git *p; @@ -1505,6 +1538,19 @@ void *read_object_file_extended(struct repository *r, return NULL; } +void *read_object_file_extended(struct repository *r, + const struct object_id *oid, + enum object_type *type, + unsigned long *size, + int lookup_replace) +{ + void *data; + obj_read_lock(); + data = do_read_object_file_extended(r, oid, type, size, lookup_replace); + obj_read_unlock(); + return data; +} + void *read_object_with_reference(const struct object_id *oid, const char *required_type_name, unsigned long *size, From patchwork Fri Jul 19 23:08:53 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matheus Tavares X-Patchwork-Id: 11050689 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 9CFD0138D for ; Fri, 19 Jul 2019 23:09:26 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 8DB3D289CB for ; Fri, 19 Jul 2019 23:09:26 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 81EE0289CF; Fri, 19 Jul 2019 23:09:26 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 14A49289CB for ; Fri, 19 Jul 2019 23:09:26 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1732571AbfGSXJY (ORCPT ); Fri, 19 Jul 2019 19:09:24 -0400 Received: from mail-qk1-f194.google.com ([209.85.222.194]:37566 "EHLO mail-qk1-f194.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728747AbfGSXJY (ORCPT ); Fri, 19 Jul 2019 19:09:24 -0400 Received: by mail-qk1-f194.google.com with SMTP id d15so24484198qkl.4 for ; Fri, 19 Jul 2019 16:09:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=usp-br.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=99i+L1MaHyEi3yva2pcCTnYL8bEWw7gen/n8t+J7Bcg=; b=ia9/gxfWZH7NaLFgMPLquyywyKhwx1tqk7buKrFKJArEcgo13fAGfBAF1jsO5M3jJ4 sRVSK4xA61EHv8OQt3fqNmTk0nZq3uDwsJowOhDViA3fkhVUV5frvLRp/9BIm1Gop0g3 ALImoibjqDiOJEaEid4pAfeWd9Bqd1yjqHfVCaVs63zBxuzkF01ywaozhW1J77FaojxJ Uamgxp3c187t6LFhoN6LUj/8FnfXDytYHWWWoY+L1dnaNb08N7m3TVkeSyZkrnqJgkn6 1xiOhJ1NgoDmDjkNzq888oCLhefcLqIH4xm7ZIKRi7t44r7g1bh3unx5lE2oBjM3wz32 9BNQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=99i+L1MaHyEi3yva2pcCTnYL8bEWw7gen/n8t+J7Bcg=; b=Ji8g1qgRoBPFaCrop/mQNqpWAGZ+QG2361vw4+9lT6cFkCZEHPYwHN+PolZIM2o20x svLYRhjpNC9Ue6Kz6yBkNuU4yF4vdr6bVcd+X7QdKM/bojsCWlVwZ/4U81KUZAHX0ges Eeyb/odSUxTRmu1J0UHGvvXt6F2DwmAIIxhmcQmxFanQe7tE6jU2Nw5EjXmuAvlWlyEG mrA/MrpMkvLzJqLjOTfQ3VsLfjeE4SG5Cf5WmW+vzhlijbcCA5ynrjj8Cc6JF63n57u+ P4EYn9wrGHWOLA3jXzP4SiNnxIDZEZ8SSbCex8/OuhfZ3+5XSHxtpLEeH5iWoEfaTebk X28g== X-Gm-Message-State: APjAAAUCdMi28rE4ijVJ1D9fkkSnCLkoD3Y+Ulk/rDTEJdaLaP8qBEd4 avIMus3amoH7GudSW4v/+CcrP5pQnKM= X-Google-Smtp-Source: APXvYqyf53J2L+oQn4Y+0pceRUGmRLlOOzpmGrGJgoWNedcqBMsalxW2NDARFwsTyF/O3+3nY8vxPg== X-Received: by 2002:a05:620a:632:: with SMTP id 18mr38915489qkv.205.1563577763302; Fri, 19 Jul 2019 16:09:23 -0700 (PDT) Received: from localhost.localdomain ([170.82.202.216]) by smtp.gmail.com with ESMTPSA id s127sm14469035qkd.107.2019.07.19.16.09.19 (version=TLS1_3 cipher=AEAD-AES256-GCM-SHA384 bits=256/256); Fri, 19 Jul 2019 16:09:22 -0700 (PDT) From: Matheus Tavares To: git@vger.kernel.org Cc: Christian Couder , Olga Telezhnaya , =?utf-8?b?Tmd1eeG7hW4gVGjDoWkg?= =?utf-8?b?Tmfhu41jIER1eQ==?= , kernel-usp@googlegroups.com, Junio C Hamano , Stefan Beller , "brian m. carlson" , Brandon Williams , Jeff King Subject: [GSoC][RFC WIP PATCH 2/3] grep: remove locks on object reading operations Date: Fri, 19 Jul 2019 20:08:53 -0300 Message-Id: <10005e2838ef3bc219b4a775de522339046db318.1563570204.git.matheus.bernardino@usp.br> X-Mailer: git-send-email 2.22.0 In-Reply-To: References: MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP git-grep has locks around the object reading operations it performs. But these have an internal lock now, which can be enabled through enable_obj_read_lock(). So let's use it and drop git-grep's ones. Signed-off-by: Matheus Tavares --- builtin/grep.c | 18 +++--------------- grep.c | 3 --- 2 files changed, 3 insertions(+), 18 deletions(-) diff --git a/builtin/grep.c b/builtin/grep.c index 580fd38f41..682e2461d0 100644 --- a/builtin/grep.c +++ b/builtin/grep.c @@ -206,6 +206,7 @@ static void start_threads(struct grep_opt *opt) pthread_cond_init(&cond_write, NULL); pthread_cond_init(&cond_result, NULL); grep_use_locks = 1; + enable_obj_read_lock(); for (i = 0; i < ARRAY_SIZE(todo); i++) { strbuf_init(&todo[i].out, 0); @@ -263,6 +264,7 @@ static int wait_all(void) pthread_cond_destroy(&cond_write); pthread_cond_destroy(&cond_result); grep_use_locks = 0; + disable_obj_read_lock(); return hit; } @@ -295,16 +297,6 @@ static int grep_cmd_config(const char *var, const char *value, void *cb) return st; } -static void *lock_and_read_oid_file(const struct object_id *oid, enum object_type *type, unsigned long *size) -{ - void *data; - - grep_read_lock(); - data = read_object_file(oid, type, size); - grep_read_unlock(); - return data; -} - static int grep_oid(struct grep_opt *opt, const struct object_id *oid, const char *filename, int tree_name_len, const char *path) @@ -457,10 +449,8 @@ static int grep_submodule(struct grep_opt *opt, object = parse_object_or_die(oid, oid_to_hex(oid)); - grep_read_lock(); data = read_object_with_reference(&object->oid, tree_type, &size, NULL); - grep_read_unlock(); if (!data) die(_("unable to read tree (%s)"), oid_to_hex(&object->oid)); @@ -585,7 +575,7 @@ static int grep_tree(struct grep_opt *opt, const struct pathspec *pathspec, void *data; unsigned long size; - data = lock_and_read_oid_file(&entry.oid, &type, &size); + data = read_object_file(&entry.oid, &type, &size); if (!data) die(_("unable to read tree (%s)"), oid_to_hex(&entry.oid)); @@ -622,10 +612,8 @@ static int grep_object(struct grep_opt *opt, const struct pathspec *pathspec, struct strbuf base; int hit, len; - grep_read_lock(); data = read_object_with_reference(&obj->oid, tree_type, &size, NULL); - grep_read_unlock(); if (!data) die(_("unable to read tree (%s)"), oid_to_hex(&obj->oid)); diff --git a/grep.c b/grep.c index f7c3a5803e..984a9e99cf 100644 --- a/grep.c +++ b/grep.c @@ -2109,10 +2109,7 @@ static int grep_source_load_oid(struct grep_source *gs) { enum object_type type; - grep_read_lock(); gs->buf = read_object_file(gs->identifier, &type, &gs->size); - grep_read_unlock(); - if (!gs->buf) return error(_("'%s': unable to read %s"), gs->name, From patchwork Fri Jul 19 23:08:54 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Matheus Tavares X-Patchwork-Id: 11050691 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id B7559138D for ; Fri, 19 Jul 2019 23:09:29 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id A748B289CB for ; Fri, 19 Jul 2019 23:09:29 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 9BC11289CF; Fri, 19 Jul 2019 23:09:29 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id BB581289CB for ; Fri, 19 Jul 2019 23:09:28 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1732592AbfGSXJ1 (ORCPT ); Fri, 19 Jul 2019 19:09:27 -0400 Received: from mail-qk1-f195.google.com ([209.85.222.195]:43702 "EHLO mail-qk1-f195.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728747AbfGSXJ1 (ORCPT ); Fri, 19 Jul 2019 19:09:27 -0400 Received: by mail-qk1-f195.google.com with SMTP id m14so24434773qka.10 for ; Fri, 19 Jul 2019 16:09:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=usp-br.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=JyZ2cI6wgRNSkeIu9CJtg1CAoXxY2w1uMLweLnF5fp8=; b=WCFK7BjBWzj+sxkFxPITg4QkNmsAUmqbch4e9PGZn7uxwGKkYVDA8CbDVN3PmFP/4D q9IoFhjL1/sA5SCOXPI2IoI46t7Z+yHw3tbf6an3vS1QdLGqEX091gX2a87pAKCXC2jh U1XFga1FPHYmydu4Tn07/E7mVrsJy6atsY3VycIVQf0kI9+Jt7LSflonFQ3ufvoXtzSv ZslxplHY0x36F31lKPYsmaG4eMBReb0fh9X3K90Yw66IH7hfxHy34VMD6wBaJnl1H3FI Ymq3I4NJ0uhYN+yKgjPDDbDX4Wl6ghlZOi0XCsUpR+0MganTheJgJsTCFwjNh2Gn1IAz rQAA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=JyZ2cI6wgRNSkeIu9CJtg1CAoXxY2w1uMLweLnF5fp8=; b=uRDz7hGIef3z02pH0oZkrdn8OeyfqRyM9+0FOuPSTp+2YHuSnTHDVXofXUB+Mh+IuE InInLciv828OJaHCayoANGkjoboLOMIIbCR6XVaoilSjyMgxFDvZF4CKl6lmTYHEfPF/ M/o9k44xwjGmpl6LK6yuof4CPV3a7aH2j5BDnOuDZ5aA7WFTeBm/Xg81zh7xYP5oeCfL RrGOdL1QLQnfwq4pocVRPeiproNsBi61ML/w+9cbdQEqNkwHPVN+btYEStiieJ3s9Fg4 ccuGAfuabinc4Apwhf5Uk3iRPo3Gvy1PhE0F+Qh1tACdM2P3BkFHon3+CuPHmo+NoddG 0mjg== X-Gm-Message-State: APjAAAUYCFT1QOTpHJTunXn8oe5SSYRfH7an7o4t1T9gzg0Y6c4LVU5w VZ/JXv51YjzaKenE3SXFHV+j38Vr+rU= X-Google-Smtp-Source: APXvYqxL5FzCp7iI89v04uo9Gy6NP6vjXNPoP3wAl2Qk5ecpya7CpSre2MAdH3Q5/QKX4s3JncE0XA== X-Received: by 2002:a37:a94:: with SMTP id 142mr34659143qkk.89.1563577766160; Fri, 19 Jul 2019 16:09:26 -0700 (PDT) Received: from localhost.localdomain ([170.82.202.216]) by smtp.gmail.com with ESMTPSA id s127sm14469035qkd.107.2019.07.19.16.09.23 (version=TLS1_3 cipher=AEAD-AES256-GCM-SHA384 bits=256/256); Fri, 19 Jul 2019 16:09:25 -0700 (PDT) From: Matheus Tavares To: git@vger.kernel.org Cc: Christian Couder , Olga Telezhnaya , =?utf-8?b?Tmd1eeG7hW4gVGjDoWkg?= =?utf-8?b?Tmfhu41jIER1eQ==?= , kernel-usp@googlegroups.com, Jeff King , Junio C Hamano , Brandon Williams Subject: [GSoC][RFC WIP PATCH 3/3] grep: re-enable threads in non-worktree case Date: Fri, 19 Jul 2019 20:08:54 -0300 Message-Id: X-Mailer: git-send-email 2.22.0 In-Reply-To: References: MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP They were disabled at 53b8d93 ("grep: disable threading in non-worktree case", 12-12-2011), due to observable perfomance drops. But now that zlib inflation can be performed in parallel, we can regain the speedup. Grep'ing 'abcd[02]' ("Regex 1") and '(static|extern) (int|double) \*' ("Regex 2") at chromium's repository[1] I got: Threads | Regex 1 | Regex 2 ---------|------------|----------- 1 | 17.5815s | 21.7217s 2 | 9.7983s | 11.3965s 8 | 6.3097s | 6.9667s These are all means of 30 executions after 2 warmup runs. All tests were executed on a i7-7700HQ with 16GB of RAM and SSD. [1]: chromium’s repo at commit 03ae96f (“Add filters testing at DSF=2”, 04-06-2019), after a 'git gc' execution. Signed-off-by: Matheus Tavares --- builtin/grep.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/builtin/grep.c b/builtin/grep.c index 682e2461d0..9309dea833 100644 --- a/builtin/grep.c +++ b/builtin/grep.c @@ -1046,7 +1046,7 @@ int cmd_grep(int argc, const char **argv, const char *prefix) pathspec.recursive = 1; pathspec.recurse_submodules = !!recurse_submodules; - if (list.nr || cached || show_in_pager) { + if (show_in_pager) { if (num_threads > 1) warning(_("invalid option combination, ignoring --threads")); num_threads = 1;