From patchwork Mon Apr 24 22:20:10 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Taylor Blau X-Patchwork-Id: 13222637 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4BDD5C77B61 for ; Mon, 24 Apr 2023 22:20:21 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233151AbjDXWUU (ORCPT ); Mon, 24 Apr 2023 18:20:20 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:38872 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229514AbjDXWUQ (ORCPT ); Mon, 24 Apr 2023 18:20:16 -0400 Received: from mail-yb1-xb35.google.com (mail-yb1-xb35.google.com [IPv6:2607:f8b0:4864:20::b35]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 8130B6185 for ; Mon, 24 Apr 2023 15:20:13 -0700 (PDT) Received: by mail-yb1-xb35.google.com with SMTP id 3f1490d57ef6-b98434b5311so6993815276.1 for ; Mon, 24 Apr 2023 15:20:13 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ttaylorr-com.20221208.gappssmtp.com; s=20221208; t=1682374812; x=1684966812; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=I1WIlbyh5VPpGVg7Ssl9PuHGafr3ZYUMF2sL7AwCXDU=; b=XON0J7pRSZ2/cSXk37crkTJjpKp+fIE5BMXjdNKeh9CwFNSpdjm1Ak3lkuVDix+iXa b1xKW65i+G9aW74PkoopTqnl58vGJj4GHevOV01Qi7hFjn+913eslReBXjMviez0kT6N ZSfkUjq0XRDmiNQ2q7XdudZxC+mSBKCZ49wsRrwk5FGaW3yrmBLc9JhGDngjWZO4oDL5 trqNqEB6Lu4VXMI64+jg3L76NwZsGLlHAP3CovzK0VTFSqplYiOdGK72/mR71HMWeoG0 nTgYMh/mM3/D8UUQOkZd5wMVRX9nZ8sefCHqi1vxErR59C5no4ZTjxpt0tuqm2d93L5B dRug== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1682374812; x=1684966812; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=I1WIlbyh5VPpGVg7Ssl9PuHGafr3ZYUMF2sL7AwCXDU=; b=RtMK/J4CYrLSCM+qCehC6S00Zv111PVKLqX2lQqVG8RWQ6ZJP07ni0I24Vz9mEyGWt Uan41oLf3PuuWFzAcomxIgQrX4jJn9joS4l2TdjQ6BOFbioYJHnQnV9o3xxZ7nGR32D6 rZ3iYbRP3VNPvONYrqNROPh1nckoW/UWkE/3FOzX7AdxNgxAexoElvfu1+tGJXVc+fDB 93L66cx4IO4HIyyFwh5qdz2oA1CaKyYr70FiSh/3xl0U+jg+SFCsfckL7gUvDC2Hr5lq fnW9xx3uoLK/P3LTcPP3TLzutjcWM/x1oiymSd/vBJmgvbYxoE2eIajlLUfcXvBWh8lW pzVQ== X-Gm-Message-State: AAQBX9cacl2QWS4XedZmHZW7Q2pHJadyIEX2NhFF+ni2S79mpeOGEKha YwTWgedRoOEqM2oITLaiuJ0HemPiGYkdgIET7cdFjQ== X-Google-Smtp-Source: AKy350ZHS9Y+DgyIuHwIq68FrKBlj2G73WdHuZcdUSWQXCj8emH1jiNTVfkfB+A8jfVcmK3GU7z4JA== X-Received: by 2002:a25:d049:0:b0:b8f:9b8:ceb7 with SMTP id h70-20020a25d049000000b00b8f09b8ceb7mr11130447ybg.44.1682374812577; Mon, 24 Apr 2023 15:20:12 -0700 (PDT) Received: from localhost (104-178-186-189.lightspeed.milwwi.sbcglobal.net. [104.178.186.189]) by smtp.gmail.com with ESMTPSA id p71-20020a0de64a000000b00552e1d1ac1esm3207823ywe.13.2023.04.24.15.20.12 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 24 Apr 2023 15:20:12 -0700 (PDT) Date: Mon, 24 Apr 2023 18:20:10 -0400 From: Taylor Blau To: git@vger.kernel.org Cc: Jeff King , Chris Torek , Junio C Hamano , Jeff Hostetler , =?utf-8?b?UmVuw6k=?= Scharfe Subject: [PATCH v3 1/6] string-list: multi-delimiter `string_list_split_in_place()` Message-ID: <59d3e778b6c80b6bd10c1126cdc3f7e86558f3e7.1682374789.git.me@ttaylorr.com> References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Enhance `string_list_split_in_place()` to accept multiple characters as delimiters instead of a single character. Instead of using `strchr(2)` to locate the first occurrence of the given delimiter character, `string_list_split_in_place_multi()` uses `strcspn(2)` to move past the initial segment of characters comprised of any characters in the delimiting set. When only a single delimiting character is provided, `strpbrk(2)` (which is implemented with `strcspn(2)`) has equivalent performance to `strchr(2)`. Modern `strcspn(2)` implementations treat an empty delimiter or the singleton delimiter as a special case and fall back to calling strchrnul(). Both glibc[1] and musl[2] implement `strcspn(2)` this way. This change is one step to removing `strtok(2)` from the tree. Note that `string_list_split_in_place()` is not a strict replacement for `strtok()`, since it will happily turn sequential delimiter characters into empty entries in the resulting string_list. For example: string_list_split_in_place(&xs, "foo:;:bar:;:baz", ":;", -1) would yield a string list of: ["foo", "", "", "bar", "", "", "baz"] Callers that wish to emulate the behavior of strtok(2) more directly should call `string_list_remove_empty_items()` after splitting. To avoid regressions for the new multi-character delimter cases, update t0063 in this patch as well. [1]: https://sourceware.org/git/?p=glibc.git;a=blob;f=string/strcspn.c;hb=glibc-2.37#l35 [2]: https://git.musl-libc.org/cgit/musl/tree/src/string/strcspn.c?h=v1.2.3#n11 Signed-off-by: Taylor Blau --- builtin/gc.c | 4 +-- diff.c | 2 +- notes.c | 2 +- refs/packed-backend.c | 2 +- string-list.c | 4 +-- string-list.h | 2 +- t/helper/test-string-list.c | 4 +-- t/t0063-string-list.sh | 51 +++++++++++++++++++++++++++++++++++++ 8 files changed, 61 insertions(+), 10 deletions(-) diff --git a/builtin/gc.c b/builtin/gc.c index edd98d35a5..f68e976704 100644 --- a/builtin/gc.c +++ b/builtin/gc.c @@ -1687,11 +1687,11 @@ static int get_schedule_cmd(const char **cmd, int *is_available) if (is_available) *is_available = 0; - string_list_split_in_place(&list, testing, ',', -1); + string_list_split_in_place(&list, testing, ",", -1); for_each_string_list_item(item, &list) { struct string_list pair = STRING_LIST_INIT_NODUP; - if (string_list_split_in_place(&pair, item->string, ':', 2) != 2) + if (string_list_split_in_place(&pair, item->string, ":", 2) != 2) continue; if (!strcmp(*cmd, pair.items[0].string)) { diff --git a/diff.c b/diff.c index 78b0fdd8ca..378a0248e1 100644 --- a/diff.c +++ b/diff.c @@ -134,7 +134,7 @@ static int parse_dirstat_params(struct diff_options *options, const char *params int i; if (*params_copy) - string_list_split_in_place(¶ms, params_copy, ',', -1); + string_list_split_in_place(¶ms, params_copy, ",", -1); for (i = 0; i < params.nr; i++) { const char *p = params.items[i].string; if (!strcmp(p, "changes")) { diff --git a/notes.c b/notes.c index 45fb7f22d1..eee806f626 100644 --- a/notes.c +++ b/notes.c @@ -963,7 +963,7 @@ void string_list_add_refs_from_colon_sep(struct string_list *list, char *globs_copy = xstrdup(globs); int i; - string_list_split_in_place(&split, globs_copy, ':', -1); + string_list_split_in_place(&split, globs_copy, ":", -1); string_list_remove_empty_items(&split, 0); for (i = 0; i < split.nr; i++) diff --git a/refs/packed-backend.c b/refs/packed-backend.c index 1eba1015dd..cc903baa7e 100644 --- a/refs/packed-backend.c +++ b/refs/packed-backend.c @@ -650,7 +650,7 @@ static struct snapshot *create_snapshot(struct packed_ref_store *refs) snapshot->buf, snapshot->eof - snapshot->buf); - string_list_split_in_place(&traits, p, ' ', -1); + string_list_split_in_place(&traits, p, " ", -1); if (unsorted_string_list_has_string(&traits, "fully-peeled")) snapshot->peeled = PEELED_FULLY; diff --git a/string-list.c b/string-list.c index db473f273e..5f5b60fe1c 100644 --- a/string-list.c +++ b/string-list.c @@ -301,7 +301,7 @@ int string_list_split(struct string_list *list, const char *string, } int string_list_split_in_place(struct string_list *list, char *string, - int delim, int maxsplit) + const char *delim, int maxsplit) { int count = 0; char *p = string, *end; @@ -315,7 +315,7 @@ int string_list_split_in_place(struct string_list *list, char *string, string_list_append(list, p); return count; } - end = strchr(p, delim); + end = strpbrk(p, delim); if (end) { *end = '\0'; string_list_append(list, p); diff --git a/string-list.h b/string-list.h index c7b0d5d000..77854840f7 100644 --- a/string-list.h +++ b/string-list.h @@ -270,5 +270,5 @@ int string_list_split(struct string_list *list, const char *string, * list->strdup_strings must *not* be set. */ int string_list_split_in_place(struct string_list *list, char *string, - int delim, int maxsplit); + const char *delim, int maxsplit); #endif /* STRING_LIST_H */ diff --git a/t/helper/test-string-list.c b/t/helper/test-string-list.c index 2123dda85b..63df88575c 100644 --- a/t/helper/test-string-list.c +++ b/t/helper/test-string-list.c @@ -62,7 +62,7 @@ int cmd__string_list(int argc, const char **argv) struct string_list list = STRING_LIST_INIT_NODUP; int i; char *s = xstrdup(argv[2]); - int delim = *argv[3]; + const char *delim = argv[3]; int maxsplit = atoi(argv[4]); i = string_list_split_in_place(&list, s, delim, maxsplit); @@ -111,7 +111,7 @@ int cmd__string_list(int argc, const char **argv) */ if (sb.len && sb.buf[sb.len - 1] == '\n') strbuf_setlen(&sb, sb.len - 1); - string_list_split_in_place(&list, sb.buf, '\n', -1); + string_list_split_in_place(&list, sb.buf, "\n", -1); string_list_sort(&list); diff --git a/t/t0063-string-list.sh b/t/t0063-string-list.sh index 46d4839194..1fee6d9010 100755 --- a/t/t0063-string-list.sh +++ b/t/t0063-string-list.sh @@ -18,6 +18,14 @@ test_split () { " } +test_split_in_place() { + cat >expected && + test_expect_success "split (in place) $1 at $2, max $3" " + test-tool string-list split_in_place '$1' '$2' '$3' >actual && + test_cmp expected actual + " +} + test_split "foo:bar:baz" ":" "-1" < X-Patchwork-Id: 13222638 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id E9445C7618E for ; Mon, 24 Apr 2023 22:20:23 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233113AbjDXWUW (ORCPT ); Mon, 24 Apr 2023 18:20:22 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:38914 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233080AbjDXWUU (ORCPT ); Mon, 24 Apr 2023 18:20:20 -0400 Received: from mail-yb1-xb33.google.com (mail-yb1-xb33.google.com [IPv6:2607:f8b0:4864:20::b33]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id F3BA96A59 for ; Mon, 24 Apr 2023 15:20:16 -0700 (PDT) Received: by mail-yb1-xb33.google.com with SMTP id 3f1490d57ef6-b8f549d36e8so8938580276.3 for ; Mon, 24 Apr 2023 15:20:16 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ttaylorr-com.20221208.gappssmtp.com; s=20221208; t=1682374816; x=1684966816; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=bFI2BL7bcCkF07NR9MDI1AUzu6vynoeiliNLGS3V+eE=; b=tO0GK+ZgnrvtZHy21lC9g52JCH5TIcfZkFaF1QRHVjtAgP8kYMRu+YwKIBNz6pQtXw mdb4QiQ0T22lAydmDA16ndZ2O3uwNsxc0OfGZa4xa3uuCOscHt75urbh0iBxC5VStTc6 sNCUn9K6Zwc6xEAT2XF+qHk0alvdjSgWOOittn5hQ7kE9mStHfW1Mh3ainAy/kb9/WoD V3GsVNl1dzTgQbMI+iwDK+eO5HJf+XYwSM6kpP1alBotr8ELGpA0Mr5qM9nAjjG1t+CK yp4h5x3r1v9Q4N0T0zbpBof3rV7EoV30+APWo4Q6ZFQsIYMGb5IEvtsMHSiv31jZdUlT YWjg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1682374816; x=1684966816; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=bFI2BL7bcCkF07NR9MDI1AUzu6vynoeiliNLGS3V+eE=; b=d9fzKfHw+IPpsDkoFmDuevU2JycUdWp+Nbgj56y2NXUc3SUe+udjOZjb+4yZjvetvk F7OjFxxIUigenXGjsVUa2+/Olk3GZYeGfDIZeuGaz6dZrUPBmB+OXuDUn08mPLeDAoiz FwTI1L8ZPNZRk+h4nVlXvY5kvLrRD0CTp/PcNDuTgJYYqVzSwX08BtIPDzzvRGkSYc06 1KtRUlFRhIHWoqDmJnXW0gT95LHOmUB860X9sjUWhlEoSJBPk3sv/fyoZeZxdlBrv84p dQCXYCbD92LdJr+nEoLReziKuDO2DCrHlhuTODwNiGqogne7WMJcJkzApJUyECZWSs9l 93GA== X-Gm-Message-State: AAQBX9ehD9wKInF2NcgFwvUg0lIpFJgfUlCKeVxTLsR0RFb1oldbtpSt VWoCkUADkAil+coE08KQXdbInh3QtAnA0Sy9NU8e3g== X-Google-Smtp-Source: AKy350abYgRNkSsC97cztM/xJNlK/+LA8ZnGT5/d4sYY0zEilhaaF7ZuvG4I1DW2IeJ573etFN/Uog== X-Received: by 2002:a25:a188:0:b0:b99:4af6:185d with SMTP id a8-20020a25a188000000b00b994af6185dmr8714602ybi.6.1682374815766; Mon, 24 Apr 2023 15:20:15 -0700 (PDT) Received: from localhost (104-178-186-189.lightspeed.milwwi.sbcglobal.net. [104.178.186.189]) by smtp.gmail.com with ESMTPSA id s14-20020a5b074e000000b00b993995abe4sm2395874ybq.60.2023.04.24.15.20.15 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 24 Apr 2023 15:20:15 -0700 (PDT) Date: Mon, 24 Apr 2023 18:20:14 -0400 From: Taylor Blau To: git@vger.kernel.org Cc: Jeff King , Chris Torek , Junio C Hamano , Jeff Hostetler , =?utf-8?b?UmVuw6k=?= Scharfe Subject: [PATCH v3 2/6] string-list: introduce `string_list_setlen()` Message-ID: References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org It is sometimes useful to reduce the size of a `string_list`'s list of items without having to re-allocate them. For example, doing the following: struct strbuf buf = STRBUF_INIT; struct string_list parts = STRING_LIST_INIT_NO_DUP; while (strbuf_getline(&buf, stdin) != EOF) { parts.nr = 0; string_list_split_in_place(&parts, buf.buf, ":", -1); /* ... */ } string_list_clear(&parts, 0); is preferable over calling `string_list_clear()` on every iteration of the loop. This is because `string_list_clear()` causes us free our existing `items` array. This means that every time we call `string_list_split_in_place()`, the string-list internals re-allocate the same size array. Since in the above example we do not care about the individual parts after processing each line, it is much more efficient to pretend that there aren't any elements in the `string_list` by setting `list->nr` to 0 while leaving the list of elements allocated as-is. This allows `string_list_split_in_place()` to overwrite any existing entries without needing to free and re-allocate them. However, setting `list->nr` manually is not safe in all instances. There are a couple of cases worth worrying about: - If the `string_list` is initialized with `strdup_strings`, truncating the list can lead to overwriting strings which are allocated elsewhere. If there aren't any other pointers to those strings other than the ones inside of the `items` array, they will become unreachable and leak. (We could ourselves free the truncated items between string_list->items[nr] and `list->nr`, but no present or future callers would benefit from this additional complexity). - If the given `nr` is larger than the current value of `list->nr`, we'll trick the `string_list` into a state where it thinks there are more items allocated than there actually are, which can lead to undefined behavior if we try to read or write those entries. Guard against both of these by introducing a helper function which guards assignment of `list->nr` against each of the above conditions. Co-authored-by: Jeff King Signed-off-by: Jeff King Signed-off-by: Taylor Blau --- string-list.c | 9 +++++++++ string-list.h | 10 ++++++++++ 2 files changed, 19 insertions(+) diff --git a/string-list.c b/string-list.c index 5f5b60fe1c..0f8ac117fd 100644 --- a/string-list.c +++ b/string-list.c @@ -203,6 +203,15 @@ void string_list_clear_func(struct string_list *list, string_list_clear_func_t c list->nr = list->alloc = 0; } +void string_list_setlen(struct string_list *list, size_t nr) +{ + if (list->strdup_strings) + BUG("cannot setlen a string_list which owns its entries"); + if (nr > list->nr) + BUG("cannot grow a string_list with setlen"); + list->nr = nr; +} + struct string_list_item *string_list_append_nodup(struct string_list *list, char *string) { diff --git a/string-list.h b/string-list.h index 77854840f7..122b318641 100644 --- a/string-list.h +++ b/string-list.h @@ -134,6 +134,16 @@ typedef void (*string_list_clear_func_t)(void *p, const char *str); /** Call a custom clear function on each util pointer */ void string_list_clear_func(struct string_list *list, string_list_clear_func_t clearfunc); +/* + * Set the length of a string_list to `nr`, provided that (a) `list` + * does not own its own storage, and (b) that `nr` is no larger than + * `list->nr`. + * + * Useful when "shrinking" `list` to write over existing entries that + * are no longer used without reallocating. + */ +void string_list_setlen(struct string_list *list, size_t nr); + /** * Apply `func` to each item. If `func` returns nonzero, the * iteration aborts and the return value is propagated. From patchwork Mon Apr 24 22:20:17 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Taylor Blau X-Patchwork-Id: 13222639 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 044CAC77B61 for ; Mon, 24 Apr 2023 22:20:29 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233250AbjDXWU2 (ORCPT ); Mon, 24 Apr 2023 18:20:28 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:38976 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233081AbjDXWUW (ORCPT ); Mon, 24 Apr 2023 18:20:22 -0400 Received: from mail-yw1-x112e.google.com (mail-yw1-x112e.google.com [IPv6:2607:f8b0:4864:20::112e]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B7C1561B8 for ; Mon, 24 Apr 2023 15:20:19 -0700 (PDT) Received: by mail-yw1-x112e.google.com with SMTP id 00721157ae682-54fb9384c2dso59593907b3.2 for ; Mon, 24 Apr 2023 15:20:19 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ttaylorr-com.20221208.gappssmtp.com; s=20221208; t=1682374819; x=1684966819; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=gF2sjAqaqKdtyrGJOlNZuAd3rzPeQPcWIGFZEps7Hn4=; b=muLB5P0ZxRDxgXczBVwyClVB1xXYEwxpzE8nX+XcYIcirEIP579q4xed5C4qexKINN JN1B3HekbkFCxBAFIU+e/QLWgubfbaXtpyGL2MHuOdgYIU/ckPPWqvjNLz98K5aTe9Z8 weCpodNVBm4ZmwpK4yxHmwKAlVar6/rhBUzCHPoWpW2rkAoJGRnD3n9iX8oEgAVmQKKy 7fk6lgFJuHeTMNDteQTAHDRHQRKUB349q/N/88N6ukvFhuLOwcmpDZgPXyRUMTPKvRrZ GxhZR6xdmzsdItWvU2QGCD06OyKcK10Cjz2eJD23VTchs1kZ6iP78pG59v1ncH7nwV5W y9UA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1682374819; x=1684966819; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=gF2sjAqaqKdtyrGJOlNZuAd3rzPeQPcWIGFZEps7Hn4=; b=asgC4UYY8k/Bg64p6LIE+5yTXzoxZMph9+loJlCsf9wASrJEETB570eUpxPnCKa5bA u3zsMCIFaRFLcZJ3PUo0M7PcSmu/jTKdSEKT1Kqz+FDOFqblkBqUM5xFFRi3e3rjwFYN ui36/sdxLcw2VJD7I/dxb2g8k0VwvfhQp53bTCCejzmhb7Z8UBdq76wepEGtbB47JTRe u8mp6PTQnXiJaB0L7fTJT3AReWnzUVRv0kZ4VUyEXYPwzb9g+d+1ym2sETqk1iF84n69 z0pe0ilHEf8ujVSRcgjuLRcdHTc4GbO32z/ZTZFQjb5MdpFzLbpONitLqXBA9cEQcG2o Kdew== X-Gm-Message-State: AAQBX9fXvksbD2XFhLS+V5AmaloOX2ZqyjIGGJ/yLmAtRi06aOnrWHBE WZq7UqoshVNOoyUX6xsN23R4BseNXtOc3KTtrUD9QQ== X-Google-Smtp-Source: AKy350ZQBoYG+kMsyiyu08xW7dA4dyGJlPsOMB6q7+9azEIlEllqaRdmOeZA45Ag7juliXiBPbbOrg== X-Received: by 2002:a0d:dd07:0:b0:54f:85a9:91c5 with SMTP id g7-20020a0ddd07000000b0054f85a991c5mr9040610ywe.24.1682374818806; Mon, 24 Apr 2023 15:20:18 -0700 (PDT) Received: from localhost (104-178-186-189.lightspeed.milwwi.sbcglobal.net. [104.178.186.189]) by smtp.gmail.com with ESMTPSA id n207-20020a0dcbd8000000b00555df877a4csm3168656ywd.102.2023.04.24.15.20.18 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 24 Apr 2023 15:20:18 -0700 (PDT) Date: Mon, 24 Apr 2023 18:20:17 -0400 From: Taylor Blau To: git@vger.kernel.org Cc: Jeff King , Chris Torek , Junio C Hamano , Jeff Hostetler , =?utf-8?b?UmVuw6k=?= Scharfe Subject: [PATCH v3 3/6] t/helper/test-hashmap.c: avoid using `strtok()` Message-ID: <78ecf13cb0a642c2591af20321245a77e9b0aadd.1682374789.git.me@ttaylorr.com> References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Avoid using the non-reentrant `strtok()` to separate the parts of each incoming command. Instead of replacing it with `strtok_r()`, let's instead use the more friendly pair of `string_list_split_in_place()` and `string_list_remove_empty_items()`. Signed-off-by: Taylor Blau --- t/helper/test-hashmap.c | 22 +++++++++++++++------- 1 file changed, 15 insertions(+), 7 deletions(-) diff --git a/t/helper/test-hashmap.c b/t/helper/test-hashmap.c index 36ff07bd4b..0eb0b3d49c 100644 --- a/t/helper/test-hashmap.c +++ b/t/helper/test-hashmap.c @@ -2,6 +2,7 @@ #include "git-compat-util.h" #include "hashmap.h" #include "strbuf.h" +#include "string-list.h" struct test_entry { @@ -150,6 +151,7 @@ static void perf_hashmap(unsigned int method, unsigned int rounds) */ int cmd__hashmap(int argc, const char **argv) { + struct string_list parts = STRING_LIST_INIT_NODUP; struct strbuf line = STRBUF_INIT; int icase; struct hashmap map = HASHMAP_INIT(test_entry_cmp, &icase); @@ -159,21 +161,26 @@ int cmd__hashmap(int argc, const char **argv) /* process commands from stdin */ while (strbuf_getline(&line, stdin) != EOF) { - char *cmd, *p1 = NULL, *p2 = NULL; + char *cmd, *p1, *p2; unsigned int hash = 0; struct test_entry *entry; /* break line into command and up to two parameters */ - cmd = strtok(line.buf, DELIM); + string_list_setlen(&parts, 0); + string_list_split_in_place(&parts, line.buf, DELIM, 2); + string_list_remove_empty_items(&parts, 0); + /* ignore empty lines */ - if (!cmd || *cmd == '#') + if (!parts.nr) + continue; + if (!*parts.items[0].string || *parts.items[0].string == '#') continue; - p1 = strtok(NULL, DELIM); - if (p1) { + cmd = parts.items[0].string; + p1 = parts.nr >= 1 ? parts.items[1].string : NULL; + p2 = parts.nr >= 2 ? parts.items[2].string : NULL; + if (p1) hash = icase ? strihash(p1) : strhash(p1); - p2 = strtok(NULL, DELIM); - } if (!strcmp("add", cmd) && p1 && p2) { @@ -260,6 +267,7 @@ int cmd__hashmap(int argc, const char **argv) } } + string_list_clear(&parts, 0); strbuf_release(&line); hashmap_clear_and_free(&map, struct test_entry, ent); return 0; From patchwork Mon Apr 24 22:20:20 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Taylor Blau X-Patchwork-Id: 13222640 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 485B2C7618E for ; Mon, 24 Apr 2023 22:20:30 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233179AbjDXWU3 (ORCPT ); Mon, 24 Apr 2023 18:20:29 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:39122 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233080AbjDXWU0 (ORCPT ); Mon, 24 Apr 2023 18:20:26 -0400 Received: from mail-yb1-xb30.google.com (mail-yb1-xb30.google.com [IPv6:2607:f8b0:4864:20::b30]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C4D407694 for ; Mon, 24 Apr 2023 15:20:22 -0700 (PDT) Received: by mail-yb1-xb30.google.com with SMTP id 3f1490d57ef6-b983027d0faso6743406276.0 for ; Mon, 24 Apr 2023 15:20:22 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ttaylorr-com.20221208.gappssmtp.com; s=20221208; t=1682374822; x=1684966822; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=XPHoAacekqzdeU0Yxd6Hn6hyK2B3LLTAZI3RMhSxhlM=; b=fHRW97v6f0XdUfsyjC7wA0o7FkZtVJY53qvLvBFr+/WZLIppkysFAU529KAdJz5fEj wQ0+Wzf0tWGA37E9AKeq8amnEW06AHU7sz2teFh8t2dfcTgwS5CCxNDY0rAmTMiLVBJ4 P8G6YTnsU4a1SSrCWlFwsRVH86oC0LC2R6hMfKKEE0SOCoJEeNhnCad41NEZ2k59JRD4 PnhBbJX5PJZI1Qr6fJTWZYVDGx6lWwTvdYcV6tIj2SrIB6WgUEq9t4wwort3IxbW5GWZ 29FLws+jlsBSAzZSJvYsbSu0bixQ9H/c0e5TWZaQZo8FKeEpNgogDkzcEeGL0NBOitoD +vcQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1682374822; x=1684966822; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=XPHoAacekqzdeU0Yxd6Hn6hyK2B3LLTAZI3RMhSxhlM=; b=RzGu6auTI+lZ5nAENP9mUIKooaoXWEJMQILQoxwZ2yDHl6+SXcx7p/lGLCnRX2Lbxh dkc1w3VCLd6m0Bw0Py6C2B6akdwbUpVz/U+3qtvuMnwiyCktCD7rL+zFF8Q6p4uYBfsl MPM2chpr3PsrowDewn8LsekDv/eOmU0PZp8XvuAHokPeu/1NwccRso0kyWFQH0jdgh4j pNCsBuqJkOiX3b7RMA0lJ5CLA0mRH6CNRXo8+IQIlOSSjdwUIC/EUg6Q3lHWbxMv5tvN cI3KdgIVCAE61Rk6Pf6FTm2nsgymvTBVKhMPCkyr2gPYaqd1+Uy/AJyFlLbwTYBua3jH JhMw== X-Gm-Message-State: AAQBX9eRzFvhGWPG8l+suEZAs8/Q0fXCgKXLfWuloqk6lvQXtD6JDHos OAlVnE0tM/tqCD05JY+U0IJkRhLarJuncSKZliy0yA== X-Google-Smtp-Source: AKy350aHRy098RrVq2GBENfitDVq6r9dxBMIMICNYTY0Fit/GTgRn1YEjCxvGAnVz6Nyt3IUtaCbsw== X-Received: by 2002:a25:ab84:0:b0:b8c:48b3:eaad with SMTP id v4-20020a25ab84000000b00b8c48b3eaadmr9787005ybi.47.1682374821859; Mon, 24 Apr 2023 15:20:21 -0700 (PDT) Received: from localhost (104-178-186-189.lightspeed.milwwi.sbcglobal.net. [104.178.186.189]) by smtp.gmail.com with ESMTPSA id m132-20020a25588a000000b00b7767ca747bsm3148576ybb.24.2023.04.24.15.20.21 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 24 Apr 2023 15:20:21 -0700 (PDT) Date: Mon, 24 Apr 2023 18:20:20 -0400 From: Taylor Blau To: git@vger.kernel.org Cc: Jeff King , Chris Torek , Junio C Hamano , Jeff Hostetler , =?utf-8?b?UmVuw6k=?= Scharfe Subject: [PATCH v3 4/6] t/helper/test-oidmap.c: avoid using `strtok()` Message-ID: References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Apply similar treatment as in the previous commit to remove usage of `strtok()` from the "oidmap" test helper. Signed-off-by: Taylor Blau --- t/helper/test-oidmap.c | 20 ++++++++++++++------ 1 file changed, 14 insertions(+), 6 deletions(-) diff --git a/t/helper/test-oidmap.c b/t/helper/test-oidmap.c index a7b7b38df1..5ce9eb3334 100644 --- a/t/helper/test-oidmap.c +++ b/t/helper/test-oidmap.c @@ -4,6 +4,7 @@ #include "oidmap.h" #include "setup.h" #include "strbuf.h" +#include "string-list.h" /* key is an oid and value is a name (could be a refname for example) */ struct test_entry { @@ -25,6 +26,7 @@ struct test_entry { */ int cmd__oidmap(int argc UNUSED, const char **argv UNUSED) { + struct string_list parts = STRING_LIST_INIT_NODUP; struct strbuf line = STRBUF_INIT; struct oidmap map = OIDMAP_INIT; @@ -35,19 +37,24 @@ int cmd__oidmap(int argc UNUSED, const char **argv UNUSED) /* process commands from stdin */ while (strbuf_getline(&line, stdin) != EOF) { - char *cmd, *p1 = NULL, *p2 = NULL; + char *cmd, *p1, *p2; struct test_entry *entry; struct object_id oid; /* break line into command and up to two parameters */ - cmd = strtok(line.buf, DELIM); + string_list_setlen(&parts, 0); + string_list_split_in_place(&parts, line.buf, DELIM, 2); + string_list_remove_empty_items(&parts, 0); + /* ignore empty lines */ - if (!cmd || *cmd == '#') + if (!parts.nr) + continue; + if (!*parts.items[0].string || *parts.items[0].string == '#') continue; - p1 = strtok(NULL, DELIM); - if (p1) - p2 = strtok(NULL, DELIM); + cmd = parts.items[0].string; + p1 = parts.nr >= 1 ? parts.items[1].string : NULL; + p2 = parts.nr >= 2 ? parts.items[2].string : NULL; if (!strcmp("put", cmd) && p1 && p2) { @@ -108,6 +115,7 @@ int cmd__oidmap(int argc UNUSED, const char **argv UNUSED) } } + string_list_clear(&parts, 0); strbuf_release(&line); oidmap_free(&map, 1); return 0; From patchwork Mon Apr 24 22:20:23 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Taylor Blau X-Patchwork-Id: 13222641 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 186B3C77B61 for ; Mon, 24 Apr 2023 22:20:39 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233174AbjDXWUi (ORCPT ); Mon, 24 Apr 2023 18:20:38 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:39140 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233257AbjDXWU3 (ORCPT ); Mon, 24 Apr 2023 18:20:29 -0400 Received: from mail-yb1-xb29.google.com (mail-yb1-xb29.google.com [IPv6:2607:f8b0:4864:20::b29]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 127548A52 for ; Mon, 24 Apr 2023 15:20:25 -0700 (PDT) Received: by mail-yb1-xb29.google.com with SMTP id 3f1490d57ef6-b99e0ffbabbso846123276.1 for ; Mon, 24 Apr 2023 15:20:25 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ttaylorr-com.20221208.gappssmtp.com; s=20221208; t=1682374825; x=1684966825; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=KuoeN0pAowH46l2c5tho3VSYmKzQ0EsUWMIdBTDhzDY=; b=z+lUu4rNanFPlU9nLHF/ugIo4lMrlotN0gDIj6QbWnmXz/M1w/gRFjXf65+oT12t4L FY5LaTB6ldpQuWr1Zgb3L5xQpNxuA6+XTthHOvpInNs4aOMWPpZFFxHNRtCBTeyeuNmK nGehgoauYj5UDwS+I4dxN8cm4UXpo0mBOR2QL2WDnO6iGguJY4/1r0+9hKHzRYkgaSOC 4fFqm66bf92IbQoJ9L7pC28umM7KA6RfPZQTwEucb2s/I39sKQFYYzEZuBfc09jcrn0u qrNKi3nlrXcUY8fhBfDtEmhiqVdJbm0jSM250Xo5bDAV4NuRKhVcIbH4+gwAPcza5o+O Ch6g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1682374825; x=1684966825; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=KuoeN0pAowH46l2c5tho3VSYmKzQ0EsUWMIdBTDhzDY=; b=bnmKDj76p7wuLeYXrYUjDYMSBX2/ytP+jaWv4X6O0z8zJN4CQQc1bdCxcKgL2ngkTc EzxfVaWQfyGU1bYAdgTE02u7MhcV8cOee0Umy0u4qW7WGwBDWjTv7V0FWwKzFBw4IMdH rYIKbUKCCj2d/fZxc0KcPsCq1c8KsEd0kZ/5G2uRcC7JMAG7db2efUg9cplLWdU4TbTm om8ClYz5NTJzOBZD3MrKPkN4Ex9a9jnR1hruD/jnbYgwf3lSZdz3qvnDlKAiRPfUJwbz NNBLnt9sXaHu821rfP9pf2OUQyvcsoGr94W1tyfAs1axQe+CIOxucwQl5FeLTP3/Is+C lW7g== X-Gm-Message-State: AAQBX9fTJhd3NULBzT0fgaEDAiov6xNRFof3C0RR1l/dByvZ4zvnC5+N f0+JmOiLA8wxS+yS615CjbkdA7p1r4JRcnOaIMgbXw== X-Google-Smtp-Source: AKy350ZAbGRcwIGKXeWdxT9jBt9UcY1SuVqp+HlKxpwoN60W3Yngh2sATnJqizuOcXSnf4CbQ3Z/GA== X-Received: by 2002:a25:aab2:0:b0:b8f:490c:a0db with SMTP id t47-20020a25aab2000000b00b8f490ca0dbmr11192454ybi.59.1682374824901; Mon, 24 Apr 2023 15:20:24 -0700 (PDT) Received: from localhost (104-178-186-189.lightspeed.milwwi.sbcglobal.net. [104.178.186.189]) by smtp.gmail.com with ESMTPSA id e65-20020a25e744000000b00b7767ca749dsm3151395ybh.58.2023.04.24.15.20.24 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 24 Apr 2023 15:20:24 -0700 (PDT) Date: Mon, 24 Apr 2023 18:20:23 -0400 From: Taylor Blau To: git@vger.kernel.org Cc: Jeff King , Chris Torek , Junio C Hamano , Jeff Hostetler , =?utf-8?b?UmVuw6k=?= Scharfe Subject: [PATCH v3 5/6] t/helper/test-json-writer.c: avoid using `strtok()` Message-ID: <201fcac6c41a80df78e58edd0d4dfda5f3944b8a.1682374789.git.me@ttaylorr.com> References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Apply similar treatment as in the previous commit to remove usage of `strtok()` from the "oidmap" test helper. Each of the different commands that the "json-writer" helper accepts pops the next space-delimited token from the current line and interprets it as a string, integer, or double (with the exception of the very first token, which is the command itself). To accommodate this, split the line in place by the space character, and pass the corresponding string_list to each of the specialized `get_s()`, `get_i()`, and `get_d()` functions. `get_i()` and `get_d()` are thin wrappers around `get_s()` that convert their result into the appropriate type by either calling `strtol()` or `strtod()`, respectively. In `get_s()`, we mark the token as "consumed" by incrementing the `consumed_nr` counter, indicating how many tokens we have read up to that point. Because each of these functions needs the string-list parts, the number of tokens consumed, and the line number, these three are wrapped up in to a struct representing the line state. Signed-off-by: Taylor Blau --- t/helper/test-json-writer.c | 76 +++++++++++++++++++++++-------------- 1 file changed, 48 insertions(+), 28 deletions(-) diff --git a/t/helper/test-json-writer.c b/t/helper/test-json-writer.c index 86887f5320..afe393f597 100644 --- a/t/helper/test-json-writer.c +++ b/t/helper/test-json-writer.c @@ -1,5 +1,6 @@ #include "test-tool.h" #include "json-writer.h" +#include "string-list.h" static const char *expect_obj1 = "{\"a\":\"abc\",\"b\":42,\"c\":true}"; static const char *expect_obj2 = "{\"a\":-1,\"b\":2147483647,\"c\":0}"; @@ -394,35 +395,41 @@ static int unit_tests(void) return 0; } -static void get_s(int line_nr, char **s_in) +struct line { + struct string_list *parts; + size_t consumed_nr; + int nr; +}; + +static void get_s(struct line *line, char **s_in) { - *s_in = strtok(NULL, " "); - if (!*s_in) - die("line[%d]: expected: ", line_nr); + if (line->consumed_nr > line->parts->nr) + die("line[%d]: expected: ", line->nr); + *s_in = line->parts->items[line->consumed_nr++].string; } -static void get_i(int line_nr, intmax_t *s_in) +static void get_i(struct line *line, intmax_t *s_in) { char *s; char *endptr; - get_s(line_nr, &s); + get_s(line, &s); *s_in = strtol(s, &endptr, 10); if (*endptr || errno == ERANGE) - die("line[%d]: invalid integer value", line_nr); + die("line[%d]: invalid integer value", line->nr); } -static void get_d(int line_nr, double *s_in) +static void get_d(struct line *line, double *s_in) { char *s; char *endptr; - get_s(line_nr, &s); + get_s(line, &s); *s_in = strtod(s, &endptr); if (*endptr || errno == ERANGE) - die("line[%d]: invalid float value", line_nr); + die("line[%d]: invalid float value", line->nr); } static int pretty; @@ -453,6 +460,7 @@ static char *get_trimmed_line(char *buf, int buf_size) static int scripted(void) { + struct string_list parts = STRING_LIST_INIT_NODUP; struct json_writer jw = JSON_WRITER_INIT; char buf[MAX_LINE_LENGTH]; char *line; @@ -470,66 +478,77 @@ static int scripted(void) die("expected first line to be 'object' or 'array'"); while ((line = get_trimmed_line(buf, MAX_LINE_LENGTH)) != NULL) { + struct line state = { 0 }; char *verb; char *key; char *s_value; intmax_t i_value; double d_value; - line_nr++; + state.parts = &parts; + state.nr = ++line_nr; - verb = strtok(line, " "); + /* break line into command and zero or more tokens */ + string_list_setlen(&parts, 0); + string_list_split_in_place(&parts, line, " ", -1); + string_list_remove_empty_items(&parts, 0); + + /* ignore empty lines */ + if (!parts.nr || !*parts.items[0].string) + continue; + + verb = parts.items[state.consumed_nr++].string; if (!strcmp(verb, "end")) { jw_end(&jw); } else if (!strcmp(verb, "object-string")) { - get_s(line_nr, &key); - get_s(line_nr, &s_value); + get_s(&state, &key); + get_s(&state, &s_value); jw_object_string(&jw, key, s_value); } else if (!strcmp(verb, "object-int")) { - get_s(line_nr, &key); - get_i(line_nr, &i_value); + get_s(&state, &key); + get_i(&state, &i_value); jw_object_intmax(&jw, key, i_value); } else if (!strcmp(verb, "object-double")) { - get_s(line_nr, &key); - get_i(line_nr, &i_value); - get_d(line_nr, &d_value); + get_s(&state, &key); + get_i(&state, &i_value); + get_d(&state, &d_value); jw_object_double(&jw, key, i_value, d_value); } else if (!strcmp(verb, "object-true")) { - get_s(line_nr, &key); + get_s(&state, &key); jw_object_true(&jw, key); } else if (!strcmp(verb, "object-false")) { - get_s(line_nr, &key); + get_s(&state, &key); jw_object_false(&jw, key); } else if (!strcmp(verb, "object-null")) { - get_s(line_nr, &key); + get_s(&state, &key); jw_object_null(&jw, key); } else if (!strcmp(verb, "object-object")) { - get_s(line_nr, &key); + get_s(&state, &key); jw_object_inline_begin_object(&jw, key); } else if (!strcmp(verb, "object-array")) { - get_s(line_nr, &key); + get_s(&state, &key); jw_object_inline_begin_array(&jw, key); } else if (!strcmp(verb, "array-string")) { - get_s(line_nr, &s_value); + get_s(&state, &s_value); jw_array_string(&jw, s_value); } else if (!strcmp(verb, "array-int")) { - get_i(line_nr, &i_value); + get_i(&state, &i_value); jw_array_intmax(&jw, i_value); } else if (!strcmp(verb, "array-double")) { - get_i(line_nr, &i_value); - get_d(line_nr, &d_value); + get_i(&state, &i_value); + get_d(&state, &d_value); jw_array_double(&jw, i_value, d_value); } else if (!strcmp(verb, "array-true")) @@ -552,6 +571,7 @@ static int scripted(void) printf("%s\n", jw.json.buf); jw_release(&jw); + string_list_clear(&parts, 0); return 0; } From patchwork Mon Apr 24 22:20:26 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Taylor Blau X-Patchwork-Id: 13222642 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 18AA2C7618E for ; Mon, 24 Apr 2023 22:20:40 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231503AbjDXWUj (ORCPT ); Mon, 24 Apr 2023 18:20:39 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:39110 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233080AbjDXWUa (ORCPT ); Mon, 24 Apr 2023 18:20:30 -0400 Received: from mail-yw1-x112a.google.com (mail-yw1-x112a.google.com [IPv6:2607:f8b0:4864:20::112a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id E64B8658A for ; Mon, 24 Apr 2023 15:20:28 -0700 (PDT) Received: by mail-yw1-x112a.google.com with SMTP id 00721157ae682-54f9d6eccf3so59357817b3.2 for ; Mon, 24 Apr 2023 15:20:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ttaylorr-com.20221208.gappssmtp.com; s=20221208; t=1682374828; x=1684966828; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=GgnoPLqFiqEJApQ0j372BU6r2Ik28j2cVQo16+oIyrU=; b=Shx26KdKiUv1nx7/MPj8ddFEUCnBjndBaKnx8IUETI9zY0UxemKMD9ctyiHeX7DcGp neT0toCeq0ze0QFt9726Cs5K4aNbE+5eErRk4lJ8P7XL9PEBFyqeTu/ppNLy5dADjEge yF5HS6HpuVG/fHTlxtxJuEQKYgg5fwAl+MJ7gUViiFxcK/bfGQc36gspJ6GB5yrWsT3D YbazgLzT97Hnqs34YZ5SoKfrBgQEKs8zh9LObck1/Is0GHtWylEvODIWRruqsYrb4fRz +sA1W99kzOz+WYQY6dqmDBDJ1OpnTFhruiJhnNp8ybVhNQAYPfhVnt81HR8Q1ab535zj 7uXg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1682374828; x=1684966828; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=GgnoPLqFiqEJApQ0j372BU6r2Ik28j2cVQo16+oIyrU=; b=ZsRoBeNgidD2JdNvhyRPtapsSUbmKRUTKz5YJHABBpHF8YTOlPOBVAxzccAYffV4lw xDEWq0FvVYfN/ma1mXtTN8f2n04OqrX84B/bWZyedA/wY/ZPGfC3IyEQy3lsH9whhX20 O6B1ZMxq1CCaeYkSHIs83msd2pKxHKFk+rj0d9Ckzfxcm0kEjYjZsFdgykCEWgoZUAV8 +gl+ZEQPwBnbZh0JBFvFe18EiRsYswePn9CztRT8EIZdUmac6Yf7DVd6vENkKQII8oft nDzpOeXMK1OWY2+EP0Jpdt9903fUwEQQ2XQCI9eq38YyNadlIAqePFv6ujuAKRXryLwI E48w== X-Gm-Message-State: AAQBX9cNmycvTMuMGA2Ua1YA17V7xENhXcNamaB97cpSGpIXogw2QNPq ROlIlEK0Kd+ZUxSuiEGg6uQc192o/9xb8Fh/jE9bRA== X-Google-Smtp-Source: AKy350bn8YWAwj3/A8k0c4pjX6C3UXsGkZvIUnVNwoUK5qQdaUjWrPnX+EG6V+X6I6qRRkkUTWP4+g== X-Received: by 2002:a81:a186:0:b0:54f:97b1:c908 with SMTP id y128-20020a81a186000000b0054f97b1c908mr9259185ywg.25.1682374827788; Mon, 24 Apr 2023 15:20:27 -0700 (PDT) Received: from localhost (104-178-186-189.lightspeed.milwwi.sbcglobal.net. [104.178.186.189]) by smtp.gmail.com with ESMTPSA id w13-20020a81a20d000000b0054fae5ed408sm3190041ywg.45.2023.04.24.15.20.27 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 24 Apr 2023 15:20:27 -0700 (PDT) Date: Mon, 24 Apr 2023 18:20:26 -0400 From: Taylor Blau To: git@vger.kernel.org Cc: Jeff King , Chris Torek , Junio C Hamano , Jeff Hostetler , =?utf-8?b?UmVuw6k=?= Scharfe Subject: [PATCH v3 6/6] banned.h: mark `strtok()` and `strtok_r()` as banned Message-ID: References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org `strtok()` has a couple of drawbacks that make it undesirable to have any new instances. In addition to being thread-unsafe, it also encourages confusing data flows, where `strtok()` may be called from multiple functions with its first argument as NULL, making it unclear from the immediate context which string is being tokenized. Now that we have removed all instances of `strtok()` from the tree, let's ban `strtok()` to avoid introducing new ones in the future. If new callers should arise, they are encouraged to use `string_list_split_in_place()` (and `string_list_remove_empty_items()`, if applicable). string_list_split_in_place() is not a perfect drop-in replacement for `strtok_r()`, particularly if the caller is processing a string with an arbitrary number of tokens, and wants to process each token one at a time. But there are no instances of this in Git's tree which are more well-suited to `strtok_r()` than the friendlier `string_list_in_place()`, so ban `strtok_r()`, too. Signed-off-by: Taylor Blau --- banned.h | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/banned.h b/banned.h index 6ccf46bc19..44e76bd90a 100644 --- a/banned.h +++ b/banned.h @@ -18,6 +18,10 @@ #define strncpy(x,y,n) BANNED(strncpy) #undef strncat #define strncat(x,y,n) BANNED(strncat) +#undef strtok +#define strtok(x,y) BANNED(strtok) +#undef strtok_r +#define strtok_r(x,y,z) BANNED(strtok_r) #undef sprintf #undef vsprintf