From patchwork Wed Jun 19 09:58:51 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Duy Nguyen X-Patchwork-Id: 11003835 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 8DAB614B6 for ; Wed, 19 Jun 2019 09:59:14 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 7BEB028A5D for ; Wed, 19 Jun 2019 09:59:14 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 6D16E28B1F; Wed, 19 Jun 2019 09:59:14 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 2415F28A5D for ; Wed, 19 Jun 2019 09:59:13 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727129AbfFSJ7M (ORCPT ); Wed, 19 Jun 2019 05:59:12 -0400 Received: from mail-pl1-f194.google.com ([209.85.214.194]:44431 "EHLO mail-pl1-f194.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727002AbfFSJ7L (ORCPT ); Wed, 19 Jun 2019 05:59:11 -0400 Received: by mail-pl1-f194.google.com with SMTP id t7so6997671plr.11 for ; Wed, 19 Jun 2019 02:59:11 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=EuAzPTbZZ9wCM7N6hdnFh/BQ5osCJc8bWfGheJCyw3Y=; b=cCy99+h4K/1+qfJwiekqcnHSFcqdfKfBY0fgluoNuahaaIyXRrivb+7KcGwEAJIBBD hiYAbWg91qq7CL46SOPivsfvshISr+CWFKrmdg00hZSyu+RlB1+vjQrL6EwQUw1vlBq9 EHX05dVTgs9bhBCtw/zAxD8of/6+OICu3/HvL2WhtjVWECUXYfoO2Lct/20ndn2lWx0F j6DyXZ9cw6GpRd/iY/QVkItaD3Rvdqlk6WRizKifl/opvQtx9Nx/fSfrtXeatzW1jsmB csSu4vvr9//3UIvcsKDEZZ8xtEEayU1UdeYeX3hgqT20ILd0dHs7PeeZJ4mQejshrHdu 3w3g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=EuAzPTbZZ9wCM7N6hdnFh/BQ5osCJc8bWfGheJCyw3Y=; b=Rj0J5xgDIxvq93NhaqM7/bQxvB5GU1iAr6An1OS+XG2Jd+CedzpdH/E9Z0Fq2QAoSi 11OZTnQgIiEheQRNCJqaJWpyurAoVb186xRKGrB7FQ87a4FjswlBqMxSGAKed0qmw5nR RgAaU2mA63PMmSrr98GZtoP0EN28QDgLs69Syw8YKqWuj+ELs+w7IPQscLOqJVAPyOLz 89IqcBvm1vuJxM17F3cHT9dizh7W7XaZjn4BLR2tRiWVmxEAeKKQNYhfDFggAdIHZukz LF5uPPICvdGXAQ9KBrGwloBJgUsaajeWjgidGJGDgh3KlJQI0IzsfqrYIyE7lWgpTWHQ DZCA== X-Gm-Message-State: APjAAAWBKaHsaBbHGP5oli1AVgGmz+pdW4NtaN2lFsQ9gSN+XGUrSz9y RcsrjwAOE8qaUaWYKy6D7Vk0O3Cr X-Google-Smtp-Source: APXvYqyqe9FAY3RkhU29cEL7WXn+MZ8jHaCj/XQCYXH89yuj1OuGf9gwUnbNLd+9jkNqdZVuTOQhrQ== X-Received: by 2002:a17:902:4c88:: with SMTP id b8mr7698087ple.29.1560938350812; Wed, 19 Jun 2019 02:59:10 -0700 (PDT) Received: from ash ([115.76.181.38]) by smtp.gmail.com with ESMTPSA id k22sm17971790pfg.77.2019.06.19.02.59.08 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 19 Jun 2019 02:59:10 -0700 (PDT) Received: by ash (sSMTP sendmail emulation); Wed, 19 Jun 2019 16:59:06 +0700 From: =?utf-8?b?Tmd1eeG7hW4gVGjDoWkgTmfhu41jIER1eQ==?= To: git@vger.kernel.org Cc: =?utf-8?b?Tmd1eeG7hW4gVGjDoWkgTmfhu41jIER1eQ==?= Subject: [PATCH 1/8] ls-files: add --json to dump the index Date: Wed, 19 Jun 2019 16:58:51 +0700 Message-Id: <20190619095858.30124-2-pclouds@gmail.com> X-Mailer: git-send-email 2.22.0.rc0.322.g2b0371e29a In-Reply-To: <20190619095858.30124-1-pclouds@gmail.com> References: <20190619095858.30124-1-pclouds@gmail.com> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP So far we don't have a command to basically dump the index file out, with all its glory details. Checking some info, for example, stat time, usually involves either writing new code or firing up "xxd" and decoding values by yourself. This --json is supposed to help that. It dumps the index in a human readable format but also easy to be processed with tools. And it will print almost enough info to reconstruct the index later. In this patch we only dump the main part, not extensions. But at the end of the series, the entire index is dumped. The end result could be very verbose even on a small repository such as git.git. Signed-off-by: Nguyễn Thái Ngọc Duy --- Documentation/git-ls-files.txt | 5 +++ builtin/ls-files.c | 30 +++++++++++--- cache.h | 2 + json-writer.c | 16 ++++++++ json-writer.h | 21 ++++++++++ read-cache.c | 73 +++++++++++++++++++++++++++++++++- 6 files changed, 140 insertions(+), 7 deletions(-) diff --git a/Documentation/git-ls-files.txt b/Documentation/git-ls-files.txt index 8461c0e83e..54011c8f65 100644 --- a/Documentation/git-ls-files.txt +++ b/Documentation/git-ls-files.txt @@ -60,6 +60,11 @@ OPTIONS --stage:: Show staged contents' mode bits, object name and stage number in the output. +--json:: + Dump the entire index content in JSON format. This is for + debugging purposes and the JSON structure may change from time + to time. + --directory:: If a whole directory is classified as "other", show just its name (with a trailing slash) and not its whole contents. diff --git a/builtin/ls-files.c b/builtin/ls-files.c index 7f83c9a6f2..d00f6d3074 100644 --- a/builtin/ls-files.c +++ b/builtin/ls-files.c @@ -8,6 +8,7 @@ #include "cache.h" #include "repository.h" #include "config.h" +#include "json-writer.h" #include "quote.h" #include "dir.h" #include "builtin.h" @@ -31,6 +32,7 @@ static int show_modified; static int show_killed; static int show_valid_bit; static int show_fsmonitor_bit; +static int show_json; static int line_terminator = '\n'; static int debug_mode; static int show_eol; @@ -543,6 +545,8 @@ int cmd_ls_files(int argc, const char **argv, const char *cmd_prefix) N_("show staged contents' object name in the output")), OPT_BOOL('k', "killed", &show_killed, N_("show files on the filesystem that need to be removed")), + OPT_BOOL(0, "json", &show_json, + N_("dump index content in json format")), OPT_BIT(0, "directory", &dir.flags, N_("show 'other' directories' names only"), DIR_SHOW_OTHER_DIRECTORIES), @@ -660,8 +664,12 @@ int cmd_ls_files(int argc, const char **argv, const char *cmd_prefix) /* With no flags, we default to showing the cached files */ if (!(show_stage || show_deleted || show_others || show_unmerged || - show_killed || show_modified || show_resolve_undo)) + show_killed || show_modified || show_resolve_undo || show_json)) show_cached = 1; + if (show_json && (show_stage || show_deleted || show_others || + show_unmerged || show_killed || show_modified || + show_cached || show_resolve_undo || with_tree)) + die(_("--show-json cannot be used with other --show- options, or --with-tree")); if (with_tree) { /* @@ -673,10 +681,22 @@ int cmd_ls_files(int argc, const char **argv, const char *cmd_prefix) overlay_tree_on_index(the_repository->index, with_tree, max_prefix); } - show_files(the_repository, &dir); - - if (show_resolve_undo) - show_ru_info(the_repository->index); + if (!show_json) { + show_files(the_repository, &dir); + + if (show_resolve_undo) + show_ru_info(the_repository->index); + } else { + struct json_writer jw = JSON_WRITER_INIT; + + discard_index(the_repository->index); + the_repository->index->jw = &jw; + if (repo_read_index(the_repository) < 0) + die("index file corrupt"); + puts(jw.json.buf); + the_repository->index->jw = NULL; + jw_release(&jw); + } if (ps_matched) { int bad; diff --git a/cache.h b/cache.h index bf20337ef4..84d0aeed20 100644 --- a/cache.h +++ b/cache.h @@ -326,6 +326,7 @@ static inline unsigned int canon_mode(unsigned int mode) #define UNTRACKED_CHANGED (1 << 7) #define FSMONITOR_CHANGED (1 << 8) +struct json_writer; struct split_index; struct untracked_cache; @@ -350,6 +351,7 @@ struct index_state { uint64_t fsmonitor_last_update; struct ewah_bitmap *fsmonitor_dirty; struct mem_pool *ce_mem_pool; + struct json_writer *jw; }; /* Name hashing */ diff --git a/json-writer.c b/json-writer.c index aadb9dbddc..281bc50b39 100644 --- a/json-writer.c +++ b/json-writer.c @@ -202,6 +202,22 @@ void jw_object_null(struct json_writer *jw, const char *key) strbuf_addstr(&jw->json, "null"); } +void jw_object_stat_data(struct json_writer *jw, const char *name, + const struct stat_data *sd) +{ + jw_object_inline_begin_object(jw, name); + jw_object_intmax(jw, "st_ctime.sec", sd->sd_ctime.sec); + jw_object_intmax(jw, "st_ctime.nsec", sd->sd_ctime.nsec); + jw_object_intmax(jw, "st_mtime.sec", sd->sd_mtime.sec); + jw_object_intmax(jw, "st_mtime.nsec", sd->sd_mtime.nsec); + jw_object_intmax(jw, "st_dev", sd->sd_dev); + jw_object_intmax(jw, "st_ino", sd->sd_ino); + jw_object_intmax(jw, "st_uid", sd->sd_uid); + jw_object_intmax(jw, "st_gid", sd->sd_gid); + jw_object_intmax(jw, "st_size", sd->sd_size); + jw_end(jw); +} + static void increase_indent(struct strbuf *sb, const struct json_writer *jw, int indent) diff --git a/json-writer.h b/json-writer.h index 83906b09c1..38f9c9bf68 100644 --- a/json-writer.h +++ b/json-writer.h @@ -44,6 +44,8 @@ #include "strbuf.h" +struct stat_data; + struct json_writer { /* @@ -81,6 +83,8 @@ void jw_object_true(struct json_writer *jw, const char *key); void jw_object_false(struct json_writer *jw, const char *key); void jw_object_bool(struct json_writer *jw, const char *key, int value); void jw_object_null(struct json_writer *jw, const char *key); +void jw_object_stat_data(struct json_writer *jw, const char *key, + const struct stat_data *sd); void jw_object_sub_jw(struct json_writer *jw, const char *key, const struct json_writer *value); @@ -104,4 +108,21 @@ void jw_array_inline_begin_array(struct json_writer *jw); int jw_is_terminated(const struct json_writer *jw); void jw_end(struct json_writer *jw); +/* + * These _gently versions accept NULL json_writer to reduce too much + * branching at the call site. + */ +static inline void jw_object_inline_begin_array_gently(struct json_writer *jw, + const char *name) +{ + if (jw) + jw_object_inline_begin_array(jw, name); +} + +static inline void jw_end_gently(struct json_writer *jw) +{ + if (jw) + jw_end(jw); +} + #endif /* JSON_WRITER_H */ diff --git a/read-cache.c b/read-cache.c index 4dd22f4f6e..eec030b3bb 100644 --- a/read-cache.c +++ b/read-cache.c @@ -25,6 +25,7 @@ #include "fsmonitor.h" #include "thread-utils.h" #include "progress.h" +#include "json-writer.h" /* Mask for the name length in ce_flags in the on-disk index */ @@ -1952,6 +1953,50 @@ static void *load_index_extensions(void *_data) return NULL; } +static void dump_cache_entry(struct index_state *istate, + int index, + unsigned long offset, + const struct cache_entry *ce) +{ + struct strbuf sb = STRBUF_INIT; + struct json_writer *jw = istate->jw; + + jw_array_inline_begin_object(jw); + + /* + * this is technically redundant, but it's for easier + * navigation when there hundreds of entries + */ + jw_object_intmax(jw, "id", index); + + jw_object_string(jw, "name", ce->name); + + strbuf_addf(&sb, "%06o", ce->ce_mode); + jw_object_string(jw, "mode", sb.buf); + strbuf_release(&sb); + + jw_object_intmax(jw, "flags", ce->ce_flags); + /* + * again redundant info, just so you don't have to decode + * flags values manually + */ + if (ce->ce_flags & CE_VALID) + jw_object_true(jw, "assume-unchanged"); + if (ce->ce_flags & CE_INTENT_TO_ADD) + jw_object_true(jw, "intent-to-add"); + if (ce->ce_flags & CE_SKIP_WORKTREE) + jw_object_true(jw, "skip-worktree"); + if (ce_stage(ce)) + jw_object_intmax(jw, "stage", ce_stage(ce)); + + jw_object_string(jw, "oid", oid_to_hex(&ce->oid)); + + jw_object_stat_data(jw, "stat", &ce->ce_stat_data); + jw_object_intmax(jw, "file-offset", offset); + + jw_end(jw); +} + /* * A helper function that will load the specified range of cache entries * from the memory mapped file and add them to the given index. @@ -1972,6 +2017,9 @@ static unsigned long load_cache_entry_block(struct index_state *istate, ce = create_from_disk(ce_mem_pool, istate->version, disk_ce, &consumed, previous_ce); set_index_entry(istate, i, ce); + if (istate->jw) + dump_cache_entry(istate, i, src_offset, ce); + src_offset += consumed; previous_ce = ce; } @@ -1983,6 +2031,8 @@ static unsigned long load_all_cache_entries(struct index_state *istate, { unsigned long consumed; + jw_object_inline_begin_array_gently(istate->jw, "entries"); + if (istate->version == 4) { mem_pool_init(&istate->ce_mem_pool, estimate_cache_size_from_compressed(istate->cache_nr)); @@ -1993,6 +2043,8 @@ static unsigned long load_all_cache_entries(struct index_state *istate, consumed = load_cache_entry_block(istate, istate->ce_mem_pool, 0, istate->cache_nr, mmap, src_offset, NULL); + + jw_end_gently(istate->jw); return consumed; } @@ -2120,6 +2172,7 @@ int do_read_index(struct index_state *istate, const char *path, int must_exist) size_t extension_offset = 0; int nr_threads, cpus; struct index_entry_offset_table *ieot = NULL; + int jw_pretty = 1; if (istate->initialized) return istate->cache_nr; @@ -2154,6 +2207,8 @@ int do_read_index(struct index_state *istate, const char *path, int must_exist) istate->cache_nr = ntohl(hdr->hdr_entries); istate->cache_alloc = alloc_nr(istate->cache_nr); istate->cache = xcalloc(istate->cache_alloc, sizeof(*istate->cache)); + istate->timestamp.sec = st.st_mtime; + istate->timestamp.nsec = ST_MTIME_NSEC(st); istate->initialized = 1; p.istate = istate; @@ -2176,6 +2231,20 @@ int do_read_index(struct index_state *istate, const char *path, int must_exist) if (!HAVE_THREADS) nr_threads = 1; + if (istate->jw) { + jw_object_begin(istate->jw, jw_pretty); + jw_object_intmax(istate->jw, "version", istate->version); + jw_object_string(istate->jw, "oid", oid_to_hex(&istate->oid)); + jw_object_intmax(istate->jw, "st_mtime.sec", istate->timestamp.sec); + jw_object_intmax(istate->jw, "st_mtime.nsec", istate->timestamp.nsec); + + /* + * Threading may mess up json writing. This is for + * debugging only, so performance is not a concern. + */ + nr_threads = 1; + } + if (nr_threads > 1) { extension_offset = read_eoie_extension(mmap, mmap_size); if (extension_offset) { @@ -2204,8 +2273,6 @@ int do_read_index(struct index_state *istate, const char *path, int must_exist) src_offset += load_all_cache_entries(istate, mmap, mmap_size, src_offset); } - istate->timestamp.sec = st.st_mtime; - istate->timestamp.nsec = ST_MTIME_NSEC(st); /* if we created a thread, join it otherwise load the extensions on the primary thread */ if (extension_offset) { @@ -2216,6 +2283,8 @@ int do_read_index(struct index_state *istate, const char *path, int must_exist) p.src_offset = src_offset; load_index_extensions(&p); } + jw_end_gently(istate->jw); + munmap((void *)mmap, mmap_size); /* From patchwork Wed Jun 19 09:58:52 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Duy Nguyen X-Patchwork-Id: 11003837 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id C7AE014B6 for ; Wed, 19 Jun 2019 09:59:17 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id B6DD828AC8 for ; Wed, 19 Jun 2019 09:59:17 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id AB7C528B20; Wed, 19 Jun 2019 09:59:17 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 3E94128AC8 for ; Wed, 19 Jun 2019 09:59:17 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1730826AbfFSJ7Q (ORCPT ); Wed, 19 Jun 2019 05:59:16 -0400 Received: from mail-pf1-f196.google.com ([209.85.210.196]:43007 "EHLO mail-pf1-f196.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727002AbfFSJ7Q (ORCPT ); Wed, 19 Jun 2019 05:59:16 -0400 Received: by mail-pf1-f196.google.com with SMTP id q10so9426684pff.9 for ; Wed, 19 Jun 2019 02:59:15 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=rDt20+hF1qPNA9jcKvPTyrjKGPqKOXcyZCN4lRI/Hb8=; b=NHLjkrxlLz1xGGRaP8Mnl1V6tu5LeAX80G7d+YcdEHAUPSM/To+u5oODjBWtuUw5fy +B5ORjDPoG/tdAe817IArbyngWY0pQ7yDKX4y1ibCbPiiFkqK9ApjT64Agcmk2VFGy2H 7FtqCE0+brt1IQ+D54sf61/x9nMSC4wE/EBXI2l2STAr6JZnz34W6BPY1tvQJXywJH5W 6Zx3lfXMLxIQ0FYTOaTYMbGbxRI5cKqswbcRpvIiM7dQkt9JC9coaXTqLeUNBWjmn9Np olAh54fxS4p6YUE/tJA4IY+BdR2XLnMMG0dHgJng9am7MftLdep29BdhMmLt/rqey/gm AGFg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=rDt20+hF1qPNA9jcKvPTyrjKGPqKOXcyZCN4lRI/Hb8=; b=WH+YcYHMuDX7bE7f7/lWZPsQ5OSRC+E4drGFzdf6gpzuIxnlTz1Ips8O/3ee2EnNbf fW5ApHNNfW1LnQoLfmq92jmEnj+2LzVuQgo62QrsPYnscdHYKzh3UUsxHbAAU7WCzrmz SkpXrfo+I5CJOAR9yFJCncaw/h+r5pgyJYYVHn4ODQV6esAIaWzx9LCqYI3SiRD4oxlW bV39vFVS2lsIgULsPDH0t2bSri8oQssTegvYeobbnS9u3z5lzHSrgxV7DeOyMCSGENX3 5AVn9rqaPsx69CpOOvsSGygbTRxYse4WJjUmuusm1CL/lqfEGaA4dyE1UAOQ6BqvW28P 0prQ== X-Gm-Message-State: APjAAAVipksL1Rlx1ria7+M9ZTEmhkxHGrkQm4Vt2UTdGpB8aYaFVMns UI2KC73uhwW4giq7Ug+VL92amAtp X-Google-Smtp-Source: APXvYqzZDfPBbXSQQq7pNPIMH4Rnmyykl3rK4PmkB0yZY7TyEqLXKMa8efExrz0wtziuuJN2LQsP9Q== X-Received: by 2002:a17:90a:fa07:: with SMTP id cm7mr10391202pjb.115.1560938355171; Wed, 19 Jun 2019 02:59:15 -0700 (PDT) Received: from ash ([115.76.181.38]) by smtp.gmail.com with ESMTPSA id a15sm3981099pgw.3.2019.06.19.02.59.12 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 19 Jun 2019 02:59:14 -0700 (PDT) Received: by ash (sSMTP sendmail emulation); Wed, 19 Jun 2019 16:59:11 +0700 From: =?utf-8?b?Tmd1eeG7hW4gVGjDoWkgTmfhu41jIER1eQ==?= To: git@vger.kernel.org Cc: =?utf-8?b?Tmd1eeG7hW4gVGjDoWkgTmfhu41jIER1eQ==?= Subject: [PATCH 2/8] split-index.c: dump "link" extension as json Date: Wed, 19 Jun 2019 16:58:52 +0700 Message-Id: <20190619095858.30124-3-pclouds@gmail.com> X-Mailer: git-send-email 2.22.0.rc0.322.g2b0371e29a In-Reply-To: <20190619095858.30124-1-pclouds@gmail.com> References: <20190619095858.30124-1-pclouds@gmail.com> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Signed-off-by: Nguyễn Thái Ngọc Duy --- json-writer.c | 14 ++++++++++++++ json-writer.h | 2 ++ split-index.c | 13 ++++++++++++- 3 files changed, 28 insertions(+), 1 deletion(-) diff --git a/json-writer.c b/json-writer.c index 281bc50b39..70403580ca 100644 --- a/json-writer.c +++ b/json-writer.c @@ -1,4 +1,5 @@ #include "cache.h" +#include "ewah/ewok.h" #include "json-writer.h" void jw_init(struct json_writer *jw) @@ -218,6 +219,19 @@ void jw_object_stat_data(struct json_writer *jw, const char *name, jw_end(jw); } +static void dump_ewah_one(size_t pos, void *jw) +{ + jw_array_intmax(jw, pos); +} + +void jw_object_ewah(struct json_writer *jw, const char *key, + struct ewah_bitmap *ewah) +{ + jw_object_inline_begin_array(jw, key); + ewah_each_bit(ewah, dump_ewah_one, jw); + jw_end(jw); +} + static void increase_indent(struct strbuf *sb, const struct json_writer *jw, int indent) diff --git a/json-writer.h b/json-writer.h index 38f9c9bf68..3c173647d3 100644 --- a/json-writer.h +++ b/json-writer.h @@ -85,6 +85,8 @@ void jw_object_bool(struct json_writer *jw, const char *key, int value); void jw_object_null(struct json_writer *jw, const char *key); void jw_object_stat_data(struct json_writer *jw, const char *key, const struct stat_data *sd); +void jw_object_ewah(struct json_writer *jw, const char *key, + struct ewah_bitmap *ewah); void jw_object_sub_jw(struct json_writer *jw, const char *key, const struct json_writer *value); diff --git a/split-index.c b/split-index.c index e6154e4ea9..d7b4420c92 100644 --- a/split-index.c +++ b/split-index.c @@ -1,4 +1,5 @@ #include "cache.h" +#include "json-writer.h" #include "split-index.h" #include "ewah/ewok.h" @@ -16,6 +17,7 @@ int read_link_extension(struct index_state *istate, { const unsigned char *data = data_; struct split_index *si; + unsigned long original_sz = sz; int ret; if (sz < the_hash_algo->rawsz) @@ -25,7 +27,7 @@ int read_link_extension(struct index_state *istate, data += the_hash_algo->rawsz; sz -= the_hash_algo->rawsz; if (!sz) - return 0; + goto done; si->delete_bitmap = ewah_new(); ret = ewah_read_mmap(si->delete_bitmap, data, sz); if (ret < 0) @@ -38,6 +40,15 @@ int read_link_extension(struct index_state *istate, return error("corrupt replace bitmap in link extension"); if (ret != sz) return error("garbage at the end of link extension"); +done: + if (istate->jw) { + jw_object_inline_begin_object(istate->jw, "split-index"); + jw_object_string(istate->jw, "oid", oid_to_hex(&si->base_oid)); + jw_object_ewah(istate->jw, "delete-bitmap", si->delete_bitmap); + jw_object_ewah(istate->jw, "replace-bitmap", si->replace_bitmap); + jw_object_intmax(istate->jw, "ext-size", original_sz); + jw_end(istate->jw); + } return 0; } From patchwork Wed Jun 19 09:58:53 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Duy Nguyen X-Patchwork-Id: 11003839 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 0550313AF for ; Wed, 19 Jun 2019 09:59:22 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id E3C0628A5D for ; Wed, 19 Jun 2019 09:59:21 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id D822B28B1F; Wed, 19 Jun 2019 09:59:21 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 8CA4928A5D for ; Wed, 19 Jun 2019 09:59:21 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1731298AbfFSJ7U (ORCPT ); Wed, 19 Jun 2019 05:59:20 -0400 Received: from mail-pg1-f196.google.com ([209.85.215.196]:35768 "EHLO mail-pg1-f196.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727002AbfFSJ7U (ORCPT ); Wed, 19 Jun 2019 05:59:20 -0400 Received: by mail-pg1-f196.google.com with SMTP id s27so9386220pgl.2 for ; Wed, 19 Jun 2019 02:59:19 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=M9Xv/oXxCB2MzWde2XsIAPLxNvKv0G045l+bVrKJShU=; b=IhdW/pzfGm/RSESRMIJtEA8cO2BY6Z/RnHno7TjZmxOP86H9G0ZyR6vVxTcIQh3v84 5gDf5nruRgwZRIFFrz6erZ5mIMVjxjkeRXQ2Jdb34/3vFbjR4H4zBJYCsvcFrfeold5k 8OQEn5K38PBM76pV5tL58eGyzYY00OK3L+xjelsMPQzZIHxGZ3PA3VkzCEUFBJycawGc Y+meaqY58V/L0t9UvF7qY5bd4VdbXStbie/yb2a5IGdNQLSOtwPJ52erS/efoL4ID2+y X4hxqd4qXFYnOCk3I6FdSlDqjN8o7/G5rj4Q3+TlXKIu6Xpa9GTxdjghkcpX/T6CNcIk JPNw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=M9Xv/oXxCB2MzWde2XsIAPLxNvKv0G045l+bVrKJShU=; b=m7TsCLrCSbQ/QQzfJLsi6CRYRqPzGi+Hy1Cn1LkmE9i2mwwnNU6B6DNwvW+kJm5u/m xIIWXYEE3xDmImWlkfR7yuXln0fmlK5aWAmDBLcv4PQ9dAEUT0KV8EZkrQx8+K25RlHt SNHqgr7IgLIx6LL9rFWs7QEgmVbGrDsYgLVCELHQ0DDaxgt0Pd9dT30YfIPbdZumeL0L vaVa5luMGDMtVd8PymFJv25BiQJhdY1I4q3YDZtXoWY2KyFMkqBmAsZidoUDXWmCLmfv sF3DEnel1azRErZCwsm+uKvIFW0kCS+g37YnGz8q8pJi2CUFxe3vDMVw7uD/U1EopxA0 vTGQ== X-Gm-Message-State: APjAAAUtN+8OjslPSqRK6+USxQ61owMx8UKGmh21J09tdZ44rTjUiy4V FXxobQuDoupX7kII0Nd+FPbzuuiy X-Google-Smtp-Source: APXvYqyIejlrkFzVEnWxvJFGcKR4N2hz1BdU7k2IXd07HDmQCnddP/cSb8QOAe7GWl0qAHfaJgXCRA== X-Received: by 2002:a63:1919:: with SMTP id z25mr7103156pgl.440.1560938359329; Wed, 19 Jun 2019 02:59:19 -0700 (PDT) Received: from ash ([115.76.181.38]) by smtp.gmail.com with ESMTPSA id j23sm20938801pgb.63.2019.06.19.02.59.17 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 19 Jun 2019 02:59:18 -0700 (PDT) Received: by ash (sSMTP sendmail emulation); Wed, 19 Jun 2019 16:59:15 +0700 From: =?utf-8?b?Tmd1eeG7hW4gVGjDoWkgTmfhu41jIER1eQ==?= To: git@vger.kernel.org Cc: =?utf-8?b?Tmd1eeG7hW4gVGjDoWkgTmfhu41jIER1eQ==?= Subject: [PATCH 3/8] fsmonitor.c: dump "FSMN" extension as json Date: Wed, 19 Jun 2019 16:58:53 +0700 Message-Id: <20190619095858.30124-4-pclouds@gmail.com> X-Mailer: git-send-email 2.22.0.rc0.322.g2b0371e29a In-Reply-To: <20190619095858.30124-1-pclouds@gmail.com> References: <20190619095858.30124-1-pclouds@gmail.com> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Signed-off-by: Nguyễn Thái Ngọc Duy --- fsmonitor.c | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/fsmonitor.c b/fsmonitor.c index 1dee0aded1..f6ba437255 100644 --- a/fsmonitor.c +++ b/fsmonitor.c @@ -3,6 +3,7 @@ #include "dir.h" #include "ewah/ewok.h" #include "fsmonitor.h" +#include "json-writer.h" #include "run-command.h" #include "strbuf.h" @@ -50,6 +51,14 @@ int read_fsmonitor_extension(struct index_state *istate, const void *data, } istate->fsmonitor_dirty = fsmonitor_dirty; + if (istate->jw) { + jw_object_inline_begin_object(istate->jw, "fsmonitor"); + jw_object_intmax(istate->jw, "version", hdr_version); + jw_object_intmax(istate->jw, "last-update", istate->fsmonitor_last_update); + jw_object_ewah(istate->jw, "dirty", fsmonitor_dirty); + jw_object_intmax(istate->jw, "ext-size", sz); + jw_end(istate->jw); + } trace_printf_key(&trace_fsmonitor, "read fsmonitor extension successful"); return 0; } From patchwork Wed Jun 19 09:58:54 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Duy Nguyen X-Patchwork-Id: 11003843 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id AF52114B6 for ; Wed, 19 Jun 2019 09:59:29 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 9E84F28A5D for ; Wed, 19 Jun 2019 09:59:29 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 91BEA28B1F; Wed, 19 Jun 2019 09:59:29 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 0C01D28A5D for ; Wed, 19 Jun 2019 09:59:29 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1731423AbfFSJ71 (ORCPT ); Wed, 19 Jun 2019 05:59:27 -0400 Received: from mail-pg1-f196.google.com ([209.85.215.196]:39538 "EHLO mail-pg1-f196.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1731385AbfFSJ7Y (ORCPT ); Wed, 19 Jun 2019 05:59:24 -0400 Received: by mail-pg1-f196.google.com with SMTP id 196so9377578pgc.6 for ; Wed, 19 Jun 2019 02:59:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=HoroAp5zJ/0rEl8i1rbsApCagkWrjHIBXfNIneeXe8c=; b=Ooo6APnO4ox9icuuG+pou+H6bW8hWHEOhQ8oZZyToYF9IzL7S4XoUGElN2Nzhyc0mx 99tvjkDFEDLpGRNuIDPdrMIyFAi5o4JoSZZogYiDCAPp0pQs8cCZo/YxbjRfb/cU9GIi yDIkvFsNCKvuMR3KWyb8t475+41W0gA1GG9+dkzjc9CxKUWxGhl8eGOyRUeW0X3fZE/+ e4kFtDwu+c0nZMdwHdobY93C6urZw/bdsqsTY3aSypxV94i8aeGGim8VyEzrNgdQlght GduI8o02WHqVOdMsVRzcEujaetxj+Hz5L4C1SqIDWOm1lOx7Tb8sZ44SfU/6B9ZKchzF IDTQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=HoroAp5zJ/0rEl8i1rbsApCagkWrjHIBXfNIneeXe8c=; b=R/VS9YQcnaTGPYuWexLxsT/9T3IuBrJTCgxPPikmzJPTmp34vK9qn1kggo15gDLcCQ hfjX+RdIEULcJ6M4vbH5SRPWIMVXk86jtGVZPh2HqAr2UxeUb26WuhjL6fq79WMhjXW+ nym5SARSUqnpCEf6gaLPfuVYPg0ab4RgGAEE4KTAn5YQI5jF6cndr5v3FqNoQAwzdN8r eJx5XtZ5HpLWSaMbhpKYL8famu9BRjrqH88fN9mnRQ5xyhUD97PSxjT6f6D5Efi5RP97 q6J7/rc8XrR/cL3xyzCs4CUabapW17SqTbUvEhdMtZyLeGRdGFoIkYdaaMX8B/sIXgPE KnXQ== X-Gm-Message-State: APjAAAX14ZpUoOanzQVtgLKmM4gbrl/8H9zOD5RF6H7jjdAlrZAPLhC9 sxDwvMoslgbj1OEUKflyeAeGoM3I X-Google-Smtp-Source: APXvYqw9l9pSz017OaQQeaCQUspUQt5eo/pY0m45/oGdYntOOb38dMa/ulvcbhhgRIh+gTqlRlqHVw== X-Received: by 2002:aa7:8d4d:: with SMTP id s13mr9855347pfe.259.1560938363812; Wed, 19 Jun 2019 02:59:23 -0700 (PDT) Received: from ash ([115.76.181.38]) by smtp.gmail.com with ESMTPSA id w132sm17717751pfd.78.2019.06.19.02.59.21 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 19 Jun 2019 02:59:23 -0700 (PDT) Received: by ash (sSMTP sendmail emulation); Wed, 19 Jun 2019 16:59:19 +0700 From: =?utf-8?b?Tmd1eeG7hW4gVGjDoWkgTmfhu41jIER1eQ==?= To: git@vger.kernel.org Cc: =?utf-8?b?Tmd1eeG7hW4gVGjDoWkgTmfhu41jIER1eQ==?= Subject: [PATCH 4/8] resolve-undo.c: dump "REUC" extension as json Date: Wed, 19 Jun 2019 16:58:54 +0700 Message-Id: <20190619095858.30124-5-pclouds@gmail.com> X-Mailer: git-send-email 2.22.0.rc0.322.g2b0371e29a In-Reply-To: <20190619095858.30124-1-pclouds@gmail.com> References: <20190619095858.30124-1-pclouds@gmail.com> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Signed-off-by: Nguyễn Thái Ngọc Duy --- read-cache.c | 2 +- resolve-undo.c | 36 +++++++++++++++++++++++++++++++++++- resolve-undo.h | 4 +++- 3 files changed, 39 insertions(+), 3 deletions(-) diff --git a/read-cache.c b/read-cache.c index eec030b3bb..3b5c63f53a 100644 --- a/read-cache.c +++ b/read-cache.c @@ -1701,7 +1701,7 @@ static int read_index_extension(struct index_state *istate, istate->cache_tree = cache_tree_read(data, sz); break; case CACHE_EXT_RESOLVE_UNDO: - istate->resolve_undo = resolve_undo_read(data, sz); + istate->resolve_undo = resolve_undo_read(data, sz, istate->jw); break; case CACHE_EXT_LINK: if (read_link_extension(istate, data, sz)) diff --git a/resolve-undo.c b/resolve-undo.c index 236320f179..999020bc40 100644 --- a/resolve-undo.c +++ b/resolve-undo.c @@ -1,5 +1,6 @@ #include "cache.h" #include "dir.h" +#include "json-writer.h" #include "resolve-undo.h" #include "string-list.h" @@ -49,7 +50,8 @@ void resolve_undo_write(struct strbuf *sb, struct string_list *resolve_undo) } } -struct string_list *resolve_undo_read(const char *data, unsigned long size) +struct string_list *resolve_undo_read(const char *data, unsigned long size, + struct json_writer *jw) { struct string_list *resolve_undo; size_t len; @@ -59,6 +61,11 @@ struct string_list *resolve_undo_read(const char *data, unsigned long size) resolve_undo = xcalloc(1, sizeof(*resolve_undo)); resolve_undo->strdup_strings = 1; + if (jw) { + jw_object_inline_begin_object(jw, "resolve-undo"); + jw_object_intmax(jw, "ext-size", size); + jw_object_inline_begin_array(jw, "entries"); + } while (size) { struct string_list_item *lost; @@ -94,6 +101,33 @@ struct string_list *resolve_undo_read(const char *data, unsigned long size) size -= rawsz; data += rawsz; } + + if (jw) { + struct strbuf sb = STRBUF_INIT; + + jw_array_inline_begin_object(jw); + jw_object_string(jw, "path", lost->string); + + jw_object_inline_begin_array(jw, "mode"); + for (i = 0; i < 3; i++) { + strbuf_addf(&sb, "%06o", ui->mode[i]); + jw_array_string(jw, sb.buf); + strbuf_reset(&sb); + } + jw_end(jw); + + jw_object_inline_begin_array(jw, "oid"); + for (i = 0; i < 3; i++) + jw_array_string(jw, oid_to_hex(&ui->oid[i])); + jw_end(jw); + + jw_end(jw); + strbuf_release(&sb); + } + } + if (jw) { + jw_end(jw); /* entries */ + jw_end(jw); /* resolve-undo */ } return resolve_undo; diff --git a/resolve-undo.h b/resolve-undo.h index 2b3f0f901e..46b4e93a7e 100644 --- a/resolve-undo.h +++ b/resolve-undo.h @@ -3,6 +3,8 @@ #include "cache.h" +struct json_writer; + struct resolve_undo_info { unsigned int mode[3]; struct object_id oid[3]; @@ -10,7 +12,7 @@ struct resolve_undo_info { void record_resolve_undo(struct index_state *, struct cache_entry *); void resolve_undo_write(struct strbuf *, struct string_list *); -struct string_list *resolve_undo_read(const char *, unsigned long); +struct string_list *resolve_undo_read(const char *, unsigned long, struct json_writer *); void resolve_undo_clear_index(struct index_state *); int unmerge_index_entry_at(struct index_state *, int); void unmerge_index(struct index_state *, const struct pathspec *); From patchwork Wed Jun 19 09:58:55 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Duy Nguyen X-Patchwork-Id: 11003845 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 1D45814B6 for ; Wed, 19 Jun 2019 09:59:33 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 0DC2228A5D for ; Wed, 19 Jun 2019 09:59:33 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 0205828B1F; Wed, 19 Jun 2019 09:59:32 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 9499D28A5D for ; Wed, 19 Jun 2019 09:59:32 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1731446AbfFSJ7b (ORCPT ); Wed, 19 Jun 2019 05:59:31 -0400 Received: from mail-pl1-f195.google.com ([209.85.214.195]:45762 "EHLO mail-pl1-f195.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1731385AbfFSJ72 (ORCPT ); Wed, 19 Jun 2019 05:59:28 -0400 Received: by mail-pl1-f195.google.com with SMTP id bi6so6983340plb.12 for ; Wed, 19 Jun 2019 02:59:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=YXgZOTo5SQg9+n9h8g3TqbNw8L8gspfKOGEd0tzsdhs=; b=b0TPgqsgzj9SShqVSsqwtEiCNRcqlSJMdx7Go0iAwRUmbMVSJ3g2O5PsRHg5hwIQRZ Oq6wRn34uXRAp31/TuTr+xIL2cwhCLcOmZYwpGdDqcQOron6lUcChKezDy1Me7LX+6Ki QaZ5V69LRCm9pC0KNMfSY0RA6qS/CB65cGBNLXGK1TyFNeS1a26lOjLcRi4n8hPDc1Ns 3HBZmCYikyxgqAjOYiqfMnz91SbNUHSN4a1S2In7qIXy6rpy6kTtzgRjAhF9TFBfK8dG EfO8/mwspRE9ayh5VuJ1reWm80BzlUjxj9zXCyiytrW/la3zZIgngJpA2a177O9B0OHr raxQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=YXgZOTo5SQg9+n9h8g3TqbNw8L8gspfKOGEd0tzsdhs=; b=OLLslul7+u2HLgRvobkNDbuDu+UstDy3ks2/9Q5EtsxoCmGg3PPxHV0pkq09en6vq+ ZL+yvKeR1ZC2dXAd5v8FcMGmQmmtU2HnfaTJWjfyfYnip+6sYFM2dixbAqVWOZ3J3RcJ /4ku6iMapnmvrnZv1xoOitzBi+FjSDN59upZIcS6zDhRDqf+OLEmflKcKOWpIz0HMSy8 FN2Jtqwdiv/0u0U6X27W+6c6PLzyGGilW4TLLQQpOorEAWg2jWLUEbZu0Otf6Y6zWbyO jqPlV1hfdxJ789l1CCciKtLnGzUvwOmBGp+1ZaYRSHh0CZL7eX73swbccilMdMHSi1Hq Wb/w== X-Gm-Message-State: APjAAAXkz4LkLiYQ9ws6NKti9wOVlO2Xlh1uexXClmagK6tMIFQIxB1T 7TWCJ4gFQSfu+VUWgmwRvXJBqLjT X-Google-Smtp-Source: APXvYqwU40w1buKSuCL5koxeiQFCTnLDSvVn0ooS+zeYpPsaL2YTEDWkvGRG+oR29XrH7dXMd/dyag== X-Received: by 2002:a17:902:bb97:: with SMTP id m23mr52298335pls.141.1560938367970; Wed, 19 Jun 2019 02:59:27 -0700 (PDT) Received: from ash ([115.76.181.38]) by smtp.gmail.com with ESMTPSA id a25sm17470811pfn.1.2019.06.19.02.59.25 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 19 Jun 2019 02:59:27 -0700 (PDT) Received: by ash (sSMTP sendmail emulation); Wed, 19 Jun 2019 16:59:24 +0700 From: =?utf-8?b?Tmd1eeG7hW4gVGjDoWkgTmfhu41jIER1eQ==?= To: git@vger.kernel.org Cc: =?utf-8?b?Tmd1eeG7hW4gVGjDoWkgTmfhu41jIER1eQ==?= Subject: [PATCH 5/8] read-cache.c: dump "EOIE" extension as json Date: Wed, 19 Jun 2019 16:58:55 +0700 Message-Id: <20190619095858.30124-6-pclouds@gmail.com> X-Mailer: git-send-email 2.22.0.rc0.322.g2b0371e29a In-Reply-To: <20190619095858.30124-1-pclouds@gmail.com> References: <20190619095858.30124-1-pclouds@gmail.com> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Signed-off-by: Nguyễn Thái Ngọc Duy --- read-cache.c | 30 +++++++++++++++++++++++++++--- 1 file changed, 27 insertions(+), 3 deletions(-) diff --git a/read-cache.c b/read-cache.c index 3b5c63f53a..04863c3853 100644 --- a/read-cache.c +++ b/read-cache.c @@ -1914,7 +1914,7 @@ struct index_entry_offset_table static struct index_entry_offset_table *read_ieot_extension(const char *mmap, size_t mmap_size, size_t offset); static void write_ieot_extension(struct strbuf *sb, struct index_entry_offset_table *ieot); -static size_t read_eoie_extension(const char *mmap, size_t mmap_size); +static size_t read_eoie_extension(const char *mmap, size_t mmap_size, struct json_writer *jw); static void write_eoie_extension(struct strbuf *sb, git_hash_ctx *eoie_context, size_t offset); struct load_index_extensions @@ -2243,10 +2243,12 @@ int do_read_index(struct index_state *istate, const char *path, int must_exist) * debugging only, so performance is not a concern. */ nr_threads = 1; + /* and dump EOIE extension even with threading off */ + read_eoie_extension(mmap, mmap_size, istate->jw); } if (nr_threads > 1) { - extension_offset = read_eoie_extension(mmap, mmap_size); + extension_offset = read_eoie_extension(mmap, mmap_size, NULL); if (extension_offset) { int err; @@ -3504,7 +3506,8 @@ int should_validate_cache_entries(void) #define EOIE_SIZE (4 + GIT_SHA1_RAWSZ) /* <4-byte offset> + <20-byte hash> */ #define EOIE_SIZE_WITH_HEADER (4 + 4 + EOIE_SIZE) /* <4-byte signature> + <4-byte length> + EOIE_SIZE */ -static size_t read_eoie_extension(const char *mmap, size_t mmap_size) +static size_t read_eoie_extension(const char *mmap, size_t mmap_size, + struct json_writer *jw) { /* * The end of index entries (EOIE) extension is guaranteed to be last @@ -3548,6 +3551,12 @@ static size_t read_eoie_extension(const char *mmap, size_t mmap_size) return 0; index += sizeof(uint32_t); + if (jw) { + jw_object_inline_begin_object(jw, "end-of-index"); + jw_object_intmax(jw, "offset", offset); + jw_object_intmax(jw, "ext-size", extsize); + jw_object_inline_begin_array(jw, "extensions"); + } /* * The hash is computed over extension types and their sizes (but not * their contents). E.g. if we have "TREE" extension that is N-bytes @@ -3576,9 +3585,24 @@ static size_t read_eoie_extension(const char *mmap, size_t mmap_size) the_hash_algo->update_fn(&c, mmap + src_offset, 8); + if (jw) { + char name[5]; + + jw_array_inline_begin_object(jw); + memcpy(name, mmap + src_offset, 4); + name[4] = '\0'; + jw_object_string(jw, "name", name); + jw_object_intmax(jw, "size", extsize); + jw_end(jw); + } + src_offset += 8; src_offset += extsize; } + if (jw) { + jw_end(jw); /* extensions */ + jw_end(jw); /* end-of-index */ + } the_hash_algo->final_fn(hash, &c); if (!hasheq(hash, (const unsigned char *)index)) return 0; From patchwork Wed Jun 19 09:58:56 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Duy Nguyen X-Patchwork-Id: 11003847 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 900AD14B6 for ; Wed, 19 Jun 2019 09:59:35 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 7F33828AC8 for ; Wed, 19 Jun 2019 09:59:35 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 721E928B20; Wed, 19 Jun 2019 09:59:35 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 08E1728AC8 for ; Wed, 19 Jun 2019 09:59:35 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1731481AbfFSJ7e (ORCPT ); Wed, 19 Jun 2019 05:59:34 -0400 Received: from mail-pf1-f196.google.com ([209.85.210.196]:44294 "EHLO mail-pf1-f196.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1731385AbfFSJ7d (ORCPT ); Wed, 19 Jun 2019 05:59:33 -0400 Received: by mail-pf1-f196.google.com with SMTP id t16so9417511pfe.11 for ; Wed, 19 Jun 2019 02:59:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=UGYeAPN7w6w1sm6Sh8QryALnaq1DTY1XELy/WQasz2s=; b=oASr2Drwgq9BVInFct8/fRtT5kKcNNUt1s2EpfrDv8qUCi61NUfrhRwQlRDR1MbS9X vBomhFQwy7u8dJ1m076NUhvivu0OkHqpd/kpgbdAbyAQKymD+zLCFFOJUFByThR+97X4 NCnslc8h5RlSSoDAloaryucyMSHu9ayVNX+xaobvzwuWsKZqFAt25WPkJJXvlk80j8A9 kw/PdilII6sCaoWDcJGxyn1ieUrs6QctPGgy++tK1N70cziGxbRDj+mKShFBQVJ4ppIN RKzalmhS16ZXZBrE6ZpAKDwStvdqu9L8F5LEKZSOjuINzoTYh/mlwi1pt4sMc6PE2Seh u2rg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=UGYeAPN7w6w1sm6Sh8QryALnaq1DTY1XELy/WQasz2s=; b=i7hjHt0tQxstKAwi6vxuxy1ORbyLgqtKxYoUehWYQGyGJRk8Wr7sqe2FGMP2hgqby/ I/NKy5DPBQa/V07hx2rYvtD06E/962cHKC8G1DD+dMIb0bs1e4EUOqFLbeOO3UwfMZVQ T6A3948PMO47uEADZ2464F+TMAfZwB/AljKEh0hG2+9Z/WldLDSVks5IoTWkTuvEGkUT vcGbom/SrEJ4xAJqYeQlnSe+NbOYnDXE6yUoKU4HMTKr+UFq/hnuXaB1mO1IaCY3dzGh 16U9AvfFmLuQxF0Zd6icoF0hJppXuHj6PXkN4Y+UtQoBAGrz5JcpgZ4sv0odW7zlpkaF ZO6g== X-Gm-Message-State: APjAAAUX3X4y4W5Occ/SORU6CEeUr9b9MYwQ10NgfBD03koyXqsXcFHl Ry/oiTfImZ76oUSnFniKco1katww X-Google-Smtp-Source: APXvYqzPlGqReBJAQSKK9WWBpVG6LS1l1jDqb766gBQoYc6+efmS6lkZtLb/eF+S/MKr2ANxULc81g== X-Received: by 2002:a62:ce07:: with SMTP id y7mr9106048pfg.12.1560938373086; Wed, 19 Jun 2019 02:59:33 -0700 (PDT) Received: from ash ([115.76.181.38]) by smtp.gmail.com with ESMTPSA id a22sm21413139pfn.173.2019.06.19.02.59.29 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 19 Jun 2019 02:59:31 -0700 (PDT) Received: by ash (sSMTP sendmail emulation); Wed, 19 Jun 2019 16:59:28 +0700 From: =?utf-8?b?Tmd1eeG7hW4gVGjDoWkgTmfhu41jIER1eQ==?= To: git@vger.kernel.org Cc: =?utf-8?b?Tmd1eeG7hW4gVGjDoWkgTmfhu41jIER1eQ==?= Subject: [PATCH 6/8] read-cache.c: dump "IEOT" extension as json Date: Wed, 19 Jun 2019 16:58:56 +0700 Message-Id: <20190619095858.30124-7-pclouds@gmail.com> X-Mailer: git-send-email 2.22.0.rc0.322.g2b0371e29a In-Reply-To: <20190619095858.30124-1-pclouds@gmail.com> References: <20190619095858.30124-1-pclouds@gmail.com> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Signed-off-by: Nguyễn Thái Ngọc Duy --- read-cache.c | 34 +++++++++++++++++++++++++++++----- 1 file changed, 29 insertions(+), 5 deletions(-) diff --git a/read-cache.c b/read-cache.c index 04863c3853..200834e77e 100644 --- a/read-cache.c +++ b/read-cache.c @@ -1911,7 +1911,7 @@ struct index_entry_offset_table struct index_entry_offset entries[FLEX_ARRAY]; }; -static struct index_entry_offset_table *read_ieot_extension(const char *mmap, size_t mmap_size, size_t offset); +static struct index_entry_offset_table *read_ieot_extension(const char *mmap, size_t mmap_size, size_t offset, struct json_writer *jw); static void write_ieot_extension(struct strbuf *sb, struct index_entry_offset_table *ieot); static size_t read_eoie_extension(const char *mmap, size_t mmap_size, struct json_writer *jw); @@ -2232,6 +2232,8 @@ int do_read_index(struct index_state *istate, const char *path, int must_exist) nr_threads = 1; if (istate->jw) { + size_t off; + jw_object_begin(istate->jw, jw_pretty); jw_object_intmax(istate->jw, "version", istate->version); jw_object_string(istate->jw, "oid", oid_to_hex(&istate->oid)); @@ -2243,8 +2245,11 @@ int do_read_index(struct index_state *istate, const char *path, int must_exist) * debugging only, so performance is not a concern. */ nr_threads = 1; - /* and dump EOIE extension even with threading off */ - read_eoie_extension(mmap, mmap_size, istate->jw); + /* and dump EOIE/IOET extensions even with threading off */ + off = read_eoie_extension(mmap, mmap_size, istate->jw); + if (off) + free(read_ieot_extension(mmap, mmap_size, + off, istate->jw)); } if (nr_threads > 1) { @@ -2266,7 +2271,7 @@ int do_read_index(struct index_state *istate, const char *path, int must_exist) * to multi-thread the reading of the cache entries. */ if (extension_offset && nr_threads > 1) - ieot = read_ieot_extension(mmap, mmap_size, extension_offset); + ieot = read_ieot_extension(mmap, mmap_size, extension_offset, NULL); if (ieot) { src_offset += load_cache_entries_threaded(istate, mmap, mmap_size, nr_threads, ieot); @@ -3630,7 +3635,9 @@ static void write_eoie_extension(struct strbuf *sb, git_hash_ctx *eoie_context, #define IEOT_VERSION (1) -static struct index_entry_offset_table *read_ieot_extension(const char *mmap, size_t mmap_size, size_t offset) +static struct index_entry_offset_table *read_ieot_extension( + const char *mmap, size_t mmap_size, + size_t offset, struct json_writer *jw) { const char *index = NULL; uint32_t extsize, ext_version; @@ -3666,6 +3673,12 @@ static struct index_entry_offset_table *read_ieot_extension(const char *mmap, si error("invalid number of IEOT entries %d", nr); return NULL; } + if (jw) { + jw_object_inline_begin_object(jw, "index-entry-offsets"); + jw_object_intmax(jw, "version", ext_version); + jw_object_intmax(jw, "ext-size", extsize); + jw_object_inline_begin_array(jw, "entries"); + } ieot = xmalloc(sizeof(struct index_entry_offset_table) + (nr * sizeof(struct index_entry_offset))); ieot->nr = nr; @@ -3674,6 +3687,17 @@ static struct index_entry_offset_table *read_ieot_extension(const char *mmap, si index += sizeof(uint32_t); ieot->entries[i].nr = get_be32(index); index += sizeof(uint32_t); + + if (jw) { + jw_array_inline_begin_object(jw); + jw_object_intmax(jw, "offset", ieot->entries[i].offset); + jw_object_intmax(jw, "count", ieot->entries[i].nr); + jw_end(jw); + } + } + if (jw) { + jw_end(jw); /* entries */ + jw_end(jw); /* index-entry-offsets */ } return ieot; From patchwork Wed Jun 19 09:58:57 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Duy Nguyen X-Patchwork-Id: 11003849 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 859FB13AF for ; Wed, 19 Jun 2019 09:59:40 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 720AE28A5D for ; Wed, 19 Jun 2019 09:59:40 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 6666428B1F; Wed, 19 Jun 2019 09:59:40 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id EA3BD28A5D for ; Wed, 19 Jun 2019 09:59:39 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1731495AbfFSJ7j (ORCPT ); Wed, 19 Jun 2019 05:59:39 -0400 Received: from mail-pg1-f193.google.com ([209.85.215.193]:40843 "EHLO mail-pg1-f193.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1731385AbfFSJ7i (ORCPT ); Wed, 19 Jun 2019 05:59:38 -0400 Received: by mail-pg1-f193.google.com with SMTP id w10so4722719pgj.7 for ; Wed, 19 Jun 2019 02:59:38 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=mOwsuLV5YQw/z24XqQpVieu79p1qEgu/udmI1Zu1dec=; b=oKVjw+/92ZkrHMW4MBEPDqWza26CCizNuGylGechtV9M4KrGjlJxHu+axvQiWlXXw0 fHb0/ceOL/9sFtbhj16NzYzWnHj0FMkdMEcRcZUMQJ/63Qe/5Rxui0qoG8f4WbMXjFk+ +SNL+UfTL8ZGhWCa8th5e1BvIgMauXddX1N7ZWu+YuUjnyXirB8TNXqOpdBNYnYpZlsN y7indrRLy30wOTragGJwBV8JcA1XUgPlyWMfMIVxoQJBzV+kNGPmq0tlawauiOqbHMqD rA3NgSOCuXGN4AiuTYrme37ppr29vjJZxi+zPp6PmX6INpxm++Z4NMOqzDXjF1Hgx2Mo Q2AA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=mOwsuLV5YQw/z24XqQpVieu79p1qEgu/udmI1Zu1dec=; b=YrA9th7zs2Q2t21Vjp0JYspWDdals79QKn+K+S/hpbh0Ja6+1uCa7Ln+o7M94LCIku bSfgDVEDkU+4Cjp9WfL+C5pOz9pDuAGaYuMepiL4s5OlucXFPPaGjotM6fVYqWeXvp2B euBJaRA2YawxRaThbU61h3iMysQUPal0139nyGg4W/DoF46z6x6/m8XorRKHQVtuj9sy ZGuPaFceOqFLeVlKV0VEYtBZUkeH01VhuRAdDQCEWo/80HJm3VyZLrLY8Z4Tge2Va0x4 8aWoZ6IoM1TEkLX5SB1tDHT9/DKV+x0jlE3ALXCSj4fS1LJjBnAMBWdXKphs2xN7jiEe YCmw== X-Gm-Message-State: APjAAAVUsQrVzNEP3qOhNct0scg8rtqkerDk7j9zY8YCsj30Va3XM2By 7FbRWU6dffmO+KjlA5K5qyMfxW0t X-Google-Smtp-Source: APXvYqyeCaoNPDON528FAYN1L003HJCOt1asf4BB1n8QIZ2FEY81zFhf6xwJZhrfT7fow89GdCoZXg== X-Received: by 2002:a63:1d10:: with SMTP id d16mr6897666pgd.446.1560938377512; Wed, 19 Jun 2019 02:59:37 -0700 (PDT) Received: from ash ([115.76.181.38]) by smtp.gmail.com with ESMTPSA id u20sm15836079pgm.56.2019.06.19.02.59.35 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 19 Jun 2019 02:59:36 -0700 (PDT) Received: by ash (sSMTP sendmail emulation); Wed, 19 Jun 2019 16:59:33 +0700 From: =?utf-8?b?Tmd1eeG7hW4gVGjDoWkgTmfhu41jIER1eQ==?= To: git@vger.kernel.org Cc: =?utf-8?b?Tmd1eeG7hW4gVGjDoWkgTmfhu41jIER1eQ==?= Subject: [PATCH 7/8] cache-tree.c: dump "TREE" extension as json Date: Wed, 19 Jun 2019 16:58:57 +0700 Message-Id: <20190619095858.30124-8-pclouds@gmail.com> X-Mailer: git-send-email 2.22.0.rc0.322.g2b0371e29a In-Reply-To: <20190619095858.30124-1-pclouds@gmail.com> References: <20190619095858.30124-1-pclouds@gmail.com> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Signed-off-by: Nguyễn Thái Ngọc Duy --- cache-tree.c | 41 ++++++++++++++++++++++++++++++++++++----- cache-tree.h | 5 ++++- read-cache.c | 2 +- 3 files changed, 41 insertions(+), 7 deletions(-) diff --git a/cache-tree.c b/cache-tree.c index b13bfaf71e..fc44016fe8 100644 --- a/cache-tree.c +++ b/cache-tree.c @@ -3,6 +3,7 @@ #include "tree.h" #include "tree-walk.h" #include "cache-tree.h" +#include "json-writer.h" #include "object-store.h" #include "replace-object.h" @@ -492,7 +493,8 @@ void cache_tree_write(struct strbuf *sb, struct cache_tree *root) write_one(sb, root, "", 0); } -static struct cache_tree *read_one(const char **buffer, unsigned long *size_p) +static struct cache_tree *read_one(const char **buffer, unsigned long *size_p, + struct json_writer *jw) { const char *buf = *buffer; unsigned long size = *size_p; @@ -546,6 +548,15 @@ static struct cache_tree *read_one(const char **buffer, unsigned long *size_p) *buffer, subtree_nr); #endif + if (jw) { + if (it->entry_count >= 0) { + jw_object_string(jw, "oid", oid_to_hex(&it->oid)); + jw_object_intmax(jw, "entry_count", it->entry_count); + } else { + jw_object_null(jw, "oid"); + } + jw_object_inline_begin_array(jw, "subdirs"); + } /* * Just a heuristic -- we do not add directories that often but * we do not want to have to extend it immediately when we do, @@ -559,12 +570,18 @@ static struct cache_tree *read_one(const char **buffer, unsigned long *size_p) struct cache_tree_sub *subtree; const char *name = buf; - sub = read_one(&buf, &size); + if (jw) { + jw_array_inline_begin_object(jw); + jw_object_string(jw, "name", name); + } + sub = read_one(&buf, &size, jw); + jw_end_gently(jw); if (!sub) goto free_return; subtree = cache_tree_sub(it, name); subtree->cache_tree = sub; } + jw_end_gently(jw); if (subtree_nr != it->subtree_nr) die("cache-tree: internal error"); *buffer = buf; @@ -576,11 +593,25 @@ static struct cache_tree *read_one(const char **buffer, unsigned long *size_p) return NULL; } -struct cache_tree *cache_tree_read(const char *buffer, unsigned long size) +struct cache_tree *cache_tree_read(const char *buffer, unsigned long size, + struct json_writer *jw) { + struct cache_tree *ret; + + if (jw) { + jw_object_inline_begin_object(jw, "cache-tree"); + jw_object_intmax(jw, "ext-size", size); + jw_object_inline_begin_object(jw, "root"); + } if (buffer[0]) - return NULL; /* not the whole tree */ - return read_one(&buffer, &size); + ret = NULL; /* not the whole tree */ + else + ret = read_one(&buffer, &size, jw); + if (jw) { + jw_end(jw); /* root */ + jw_end(jw); /* cache-tree */ + } + return ret; } static struct cache_tree *cache_tree_find(struct cache_tree *it, const char *path) diff --git a/cache-tree.h b/cache-tree.h index 757bbc48bc..fc3c73284b 100644 --- a/cache-tree.h +++ b/cache-tree.h @@ -6,6 +6,8 @@ #include "tree-walk.h" struct cache_tree; +struct json_writer; + struct cache_tree_sub { struct cache_tree *cache_tree; int count; /* internally used by update_one() */ @@ -28,7 +30,8 @@ void cache_tree_invalidate_path(struct index_state *, const char *); struct cache_tree_sub *cache_tree_sub(struct cache_tree *, const char *); void cache_tree_write(struct strbuf *, struct cache_tree *root); -struct cache_tree *cache_tree_read(const char *buffer, unsigned long size); +struct cache_tree *cache_tree_read(const char *buffer, unsigned long size, + struct json_writer *jw); int cache_tree_fully_valid(struct cache_tree *); int cache_tree_update(struct index_state *, int); diff --git a/read-cache.c b/read-cache.c index 200834e77e..289705b816 100644 --- a/read-cache.c +++ b/read-cache.c @@ -1698,7 +1698,7 @@ static int read_index_extension(struct index_state *istate, { switch (CACHE_EXT(ext)) { case CACHE_EXT_TREE: - istate->cache_tree = cache_tree_read(data, sz); + istate->cache_tree = cache_tree_read(data, sz, istate->jw); break; case CACHE_EXT_RESOLVE_UNDO: istate->resolve_undo = resolve_undo_read(data, sz, istate->jw); From patchwork Wed Jun 19 09:58:58 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Duy Nguyen X-Patchwork-Id: 11003851 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 8DF9114B6 for ; Wed, 19 Jun 2019 09:59:44 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 7C12028A5D for ; Wed, 19 Jun 2019 09:59:44 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 7077728B1F; Wed, 19 Jun 2019 09:59:44 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id E4A3328A5D for ; Wed, 19 Jun 2019 09:59:43 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1731508AbfFSJ7n (ORCPT ); Wed, 19 Jun 2019 05:59:43 -0400 Received: from mail-pg1-f196.google.com ([209.85.215.196]:33237 "EHLO mail-pg1-f196.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1731385AbfFSJ7m (ORCPT ); Wed, 19 Jun 2019 05:59:42 -0400 Received: by mail-pg1-f196.google.com with SMTP id k187so9381399pga.0 for ; Wed, 19 Jun 2019 02:59:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=X31QZYE2YBKeGWyA1pEB2fxKzniHQtaARgTcTHQfvD0=; b=ZsAd5iX8slSCdcPq58YTuRpxt9Y/Nd8N2xEvtyX9QMnqu+PRpI+0kozRjkISHahcGx ZOXZ2l7WmaClz2flSD4YIugt33jxl4up0lzsxcJHZeuoNlEWknLGYN+79we9ilDNteSN 3w1Re0Bdb4YLho6Hx8E7RpynLGXZ8n0iScnRizfMNW6noBrFH3yH12+LD6koyjnYkXlo hE2x9sK9bp8DXAUtUJfaFo94yKN2pG7JFV8cVQmCuPHiy2PngICiDdz3tXm7mVJ0zjYR qY8e86DhKnx7s1+Jc6uQ683GqUF8QpPpbqsCWegNV7grUKZt1MVmaTIQPshoDKRQDNds FBxw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=X31QZYE2YBKeGWyA1pEB2fxKzniHQtaARgTcTHQfvD0=; b=EP0yaJbBhaYaQWmT8WbpaMyLqwUrqVUQ99KG3A9hrB9+FcBHOMlMQuhbuNP41IMsHN NGXgDGgwtb3t3dYcXv3sATnR3sdFaz3OvfpD42A1bkKw1FSWSy5xs39cE0ybf3FGh/7o dpIDd5iK01UQ1XD/n/uAT044ASkeA30EmGq2BICw+iZxp1XQizfPjAXxZV4Pfz2VcmOo 2JGsCKkbsMlahM7PsYPnnEYPujAyuZGVH9J0ZH4tgPjnHhVG3Y6pBllox0pK/wVeqyBt GL7T4LqRvK335riGRGRKpZoGvJ9g39bbm6ngK/9nT8YnlHBgrRlEWR9nzstEysVVx7If ux5Q== X-Gm-Message-State: APjAAAWeYctjCwRSICy8+mp/cNdZtVK9JKKkhKp/iilyQ2XmYdK1m34b U5S8ckgUe5+vgX3uiEahccK8MPxF X-Google-Smtp-Source: APXvYqwThc/NEicMteCPlCdomBO9URHEvaXm3TLGw7ZM4/YeDDfohDfOiaPa1VIm8f9kyonSmx9Ngg== X-Received: by 2002:a62:1515:: with SMTP id 21mr47587459pfv.100.1560938381835; Wed, 19 Jun 2019 02:59:41 -0700 (PDT) Received: from ash ([115.76.181.38]) by smtp.gmail.com with ESMTPSA id 201sm25107922pfz.24.2019.06.19.02.59.39 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 19 Jun 2019 02:59:41 -0700 (PDT) Received: by ash (sSMTP sendmail emulation); Wed, 19 Jun 2019 16:59:37 +0700 From: =?utf-8?b?Tmd1eeG7hW4gVGjDoWkgTmfhu41jIER1eQ==?= To: git@vger.kernel.org Cc: =?utf-8?b?Tmd1eeG7hW4gVGjDoWkgTmfhu41jIER1eQ==?= Subject: [PATCH 8/8] dir.c: dump "UNTR" extension as json Date: Wed, 19 Jun 2019 16:58:58 +0700 Message-Id: <20190619095858.30124-9-pclouds@gmail.com> X-Mailer: git-send-email 2.22.0.rc0.322.g2b0371e29a In-Reply-To: <20190619095858.30124-1-pclouds@gmail.com> References: <20190619095858.30124-1-pclouds@gmail.com> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP The big part of UNTR extension is dumped at the end instead of dumping as soon as we read it, because we actually "patch" some fields in untracked_cache_dir with EWAH bitmaps at the end. Signed-off-by: Nguyễn Thái Ngọc Duy --- dir.c | 56 ++++++++++++++++++++++++++++++++++++++++++++++++++- dir.h | 4 +++- json-writer.h | 6 ++++++ read-cache.c | 2 +- 4 files changed, 65 insertions(+), 3 deletions(-) diff --git a/dir.c b/dir.c index ba4a51c296..f389eee24a 100644 --- a/dir.c +++ b/dir.c @@ -19,6 +19,7 @@ #include "varint.h" #include "ewah/ewok.h" #include "fsmonitor.h" +#include "json-writer.h" #include "submodule-config.h" /* @@ -2826,7 +2827,42 @@ static void load_oid_stat(struct oid_stat *oid_stat, const unsigned char *data, oid_stat->valid = 1; } -struct untracked_cache *read_untracked_extension(const void *data, unsigned long sz) +static void jw_object_oid_stat(struct json_writer *jw, const char *key, + const struct oid_stat *oid_stat) +{ + jw_object_inline_begin_object(jw, key); + jw_object_bool(jw, "valid", oid_stat->valid); + jw_object_string(jw, "oid", oid_to_hex(&oid_stat->oid)); + jw_object_stat_data(jw, "stat", &oid_stat->stat); + jw_end(jw); +} + +static void jw_object_untracked_cache_dir(struct json_writer *jw, + const struct untracked_cache_dir *ucd) +{ + int i; + + jw_object_bool(jw, "valid", ucd->valid); + jw_object_bool(jw, "check-only", ucd->check_only); + jw_object_stat_data(jw, "stat", &ucd->stat_data); + jw_object_string(jw, "exclude-oid", oid_to_hex(&ucd->exclude_oid)); + jw_object_inline_begin_array(jw, "untracked"); + for (i = 0; i < ucd->untracked_nr; i++) + jw_array_string(jw, ucd->untracked[i]); + jw_end(jw); + + jw_object_inline_begin_object(jw, "dirs"); + for (i = 0; i < ucd->dirs_nr; i++) { + jw_object_inline_begin_object(jw, ucd->dirs[i]->name); + jw_object_untracked_cache_dir(jw, ucd->dirs[i]); + jw_end(jw); + } + jw_end(jw); +} + +struct untracked_cache *read_untracked_extension(const void *data, + unsigned long sz, + struct json_writer *jw) { struct untracked_cache *uc; struct read_data rd; @@ -2864,6 +2900,17 @@ struct untracked_cache *read_untracked_extension(const void *data, unsigned long uc->dir_flags = get_be32(next + ouc_offset(dir_flags)); exclude_per_dir = (const char *)next + exclude_per_dir_offset; uc->exclude_per_dir = xstrdup(exclude_per_dir); + + if (jw) { + jw_object_inline_begin_object(jw, "untracked-cache"); + jw_object_intmax(jw, "ext-size", sz); + jw_object_string(jw, "ident", ident); + jw_object_oid_stat(jw, "info/exclude", &uc->ss_info_exclude); + jw_object_oid_stat(jw, "excludes-file", &uc->ss_excludes_file); + jw_object_intmax(jw, "flags", uc->dir_flags); + jw_object_string(jw, "excludes-per-dir", uc->exclude_per_dir); + } + /* NUL after exclude_per_dir is covered by sizeof(*ouc) */ next += exclude_per_dir_offset + strlen(exclude_per_dir) + 1; if (next >= end) @@ -2905,6 +2952,12 @@ struct untracked_cache *read_untracked_extension(const void *data, unsigned long ewah_each_bit(rd.sha1_valid, read_oid, &rd); next = rd.data; + if (jw) { + jw_object_inline_begin_object(jw, "root"); + jw_object_untracked_cache_dir(jw, uc->root); + jw_end(jw); + } + done: free(rd.ucd); ewah_free(rd.valid); @@ -2915,6 +2968,7 @@ struct untracked_cache *read_untracked_extension(const void *data, unsigned long free_untracked_cache(uc); uc = NULL; } + jw_end_gently(jw); return uc; } diff --git a/dir.h b/dir.h index 680079bbe3..80efdd05c4 100644 --- a/dir.h +++ b/dir.h @@ -6,6 +6,8 @@ #include "cache.h" #include "strbuf.h" +struct json_writer; + struct dir_entry { unsigned int len; char name[FLEX_ARRAY]; /* more */ @@ -362,7 +364,7 @@ void untracked_cache_remove_from_index(struct index_state *, const char *); void untracked_cache_add_to_index(struct index_state *, const char *); void free_untracked_cache(struct untracked_cache *); -struct untracked_cache *read_untracked_extension(const void *data, unsigned long sz); +struct untracked_cache *read_untracked_extension(const void *data, unsigned long sz, struct json_writer *jw); void write_untracked_extension(struct strbuf *out, struct untracked_cache *untracked); void add_untracked_cache(struct index_state *istate); void remove_untracked_cache(struct index_state *istate); diff --git a/json-writer.h b/json-writer.h index 3c173647d3..f778e019a2 100644 --- a/json-writer.h +++ b/json-writer.h @@ -121,6 +121,12 @@ static inline void jw_object_inline_begin_array_gently(struct json_writer *jw, jw_object_inline_begin_array(jw, name); } +static inline void jw_array_inline_begin_object_gently(struct json_writer *jw) +{ + if (jw) + jw_array_inline_begin_object(jw); +} + static inline void jw_end_gently(struct json_writer *jw) { if (jw) diff --git a/read-cache.c b/read-cache.c index 289705b816..d7d9ce7260 100644 --- a/read-cache.c +++ b/read-cache.c @@ -1708,7 +1708,7 @@ static int read_index_extension(struct index_state *istate, return -1; break; case CACHE_EXT_UNTRACKED: - istate->untracked = read_untracked_extension(data, sz); + istate->untracked = read_untracked_extension(data, sz, istate->jw); break; case CACHE_EXT_FSMONITOR: read_fsmonitor_extension(istate, data, sz);