From patchwork Thu Apr 13 23:31:43 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Taylor Blau X-Patchwork-Id: 13210753 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id B1202C77B61 for ; Thu, 13 Apr 2023 23:31:48 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229733AbjDMXbr (ORCPT ); Thu, 13 Apr 2023 19:31:47 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58276 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229492AbjDMXbq (ORCPT ); Thu, 13 Apr 2023 19:31:46 -0400 Received: from mail-yb1-xb32.google.com (mail-yb1-xb32.google.com [IPv6:2607:f8b0:4864:20::b32]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 9538B3ABB for ; Thu, 13 Apr 2023 16:31:45 -0700 (PDT) Received: by mail-yb1-xb32.google.com with SMTP id h198so23710097ybg.12 for ; Thu, 13 Apr 2023 16:31:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ttaylorr-com.20221208.gappssmtp.com; s=20221208; t=1681428704; x=1684020704; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=76KQIvRDAoWE4iUGUMJRAu6V6XAytZiNEGCb/l7TpcU=; b=q/oryTdnJVMO4SxfQuUuIukvJowsix/vUeHGdwKVE7qgU5yqq/jMM8w+UHGnerNSD8 vorLNS6rm6uJu6ouNtEsbf11Y4oiXyZXAREl4RE+RkzC0DuUIW3wgUgmG+Lrk1JcWZns Q80CY/Br0/COwyh3wgoF/3Twy/q+uuqi93HxMESlzHGKnbeMXF6e4Rx0g2/kBUFq5bA3 8fNkn4nbSOIcxSTTxtFgXYkreLuzJgJKmYYUKHvR93RCSbd2yZKl1JxHQgJsBzI0rRcV +cLjKBaGfGgCjqjxfAoJG0NkPOyoVXl41sCiXoh7C2bwqbnsV2qP+qQQHb4maeonOjXL wgkQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1681428704; x=1684020704; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=76KQIvRDAoWE4iUGUMJRAu6V6XAytZiNEGCb/l7TpcU=; b=AU+8h3XODMFIQonnwjsl61T/T8g+8L1jrHAlbZ5qmknguaiL/W0mFH0h8f4U/xZ2+T aHtLcrDJ8OacENzG/gkOU/MFjdSbcPA4z5gZhZF3z47QX/0irtGOfMeF2PhtP80ze5FH /k6GX5HlldDdqIwyOs0ykur2DHQEz6vBJTOzviN3hM6FAJfh92M0IXs/Oc9unggtmO3P gngTQG0m4J77bVQAryVkZvpNEZYCswZ1WeqgjELh3hQQvrG3pjid3UEyXioNa7w3Haou Urq0pRzDAsANqFvRVXwKh81B5nx+CaTUU+LdZumb+AnoX/XIEM7zxyrPhcCSk1dRNdYY 65UQ== X-Gm-Message-State: AAQBX9fpUhNDrA89JTRVU76By1KQlocq3bYHC2zJnR5pfw6ymipSLNhK KqlEQLy+4kliDjZtGA43m8kebwx2aM6LijQr0PoVkA== X-Google-Smtp-Source: AKy350YG6hTBcbdtn9E2I6TgIjMxw4qTuE0zRaRYMH9Qw/Omee8+Oys3uDG6AmpLzvmIFs95cNfa9A== X-Received: by 2002:a25:db0a:0:b0:b8f:f5f:13e9 with SMTP id g10-20020a25db0a000000b00b8f0f5f13e9mr3678724ybf.2.1681428704601; Thu, 13 Apr 2023 16:31:44 -0700 (PDT) Received: from localhost (104-178-186-189.lightspeed.milwwi.sbcglobal.net. [104.178.186.189]) by smtp.gmail.com with ESMTPSA id s10-20020a5b074a000000b00b8f6eba39cesm61699ybq.57.2023.04.13.16.31.44 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 13 Apr 2023 16:31:44 -0700 (PDT) Date: Thu, 13 Apr 2023 19:31:43 -0400 From: Taylor Blau To: git@vger.kernel.org Cc: Jeff King , Junio C Hamano Subject: [PATCH 1/5] string-list: introduce `string_list_split_in_place_multi()` Message-ID: References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Introduce a variant of the `string_list_split_in_place()` function that takes a string of accepted delimiters. By contrast to its cousin `string_list_split_in_place()` which splits the given string at every instance of the single character `delim`, the `_multi` variant splits the given string any any character appearing in the string `delim`. Instead of using `strchr(2)` to locate the first occurrence of the given delimiter character, `string_list_split_in_place_multi()` uses `strpbrk(2)` to find the first occurrence of *any* character in the given delimiter string. Since the `_multi` variant is a generalization of the original implementation, reimplement `string_list_split_in_place()` in terms of the more general function by providing a single-character string for the list of accepted delimiters. Signed-off-by: Taylor Blau --- string-list.c | 15 ++++++++++++--- string-list.h | 6 ++++++ 2 files changed, 18 insertions(+), 3 deletions(-) diff --git a/string-list.c b/string-list.c index db473f273e1..67f9ff18904 100644 --- a/string-list.c +++ b/string-list.c @@ -300,8 +300,8 @@ int string_list_split(struct string_list *list, const char *string, } } -int string_list_split_in_place(struct string_list *list, char *string, - int delim, int maxsplit) +int string_list_split_in_place_multi(struct string_list *list, char *string, + const char *delim, int maxsplit) { int count = 0; char *p = string, *end; @@ -315,7 +315,7 @@ int string_list_split_in_place(struct string_list *list, char *string, string_list_append(list, p); return count; } - end = strchr(p, delim); + end = strpbrk(p, delim); if (end) { *end = '\0'; string_list_append(list, p); @@ -326,3 +326,12 @@ int string_list_split_in_place(struct string_list *list, char *string, } } } + +int string_list_split_in_place(struct string_list *list, char *string, + int delim, int maxsplit) +{ + char delim_s[2] = { delim, 0 }; + + return string_list_split_in_place_multi(list, string, delim_s, + maxsplit); +} diff --git a/string-list.h b/string-list.h index c7b0d5d0008..670d4fc8fb7 100644 --- a/string-list.h +++ b/string-list.h @@ -268,7 +268,13 @@ int string_list_split(struct string_list *list, const char *string, * new string_list_items point into string (which therefore must not * be modified or freed while the string_list is in use). * list->strdup_strings must *not* be set. + * + * The "_multi" variant splits the given string on any character + * appearing in "delim", and the non-"_multi" variant splits only on the + * given character. */ +int string_list_split_in_place_multi(struct string_list *list, char *string, + const char *delim, int maxsplit); int string_list_split_in_place(struct string_list *list, char *string, int delim, int maxsplit); #endif /* STRING_LIST_H */ From patchwork Thu Apr 13 23:31:46 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Taylor Blau X-Patchwork-Id: 13210754 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 58F21C77B61 for ; Thu, 13 Apr 2023 23:31:52 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229778AbjDMXbv (ORCPT ); Thu, 13 Apr 2023 19:31:51 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58388 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229958AbjDMXbu (ORCPT ); Thu, 13 Apr 2023 19:31:50 -0400 Received: from mail-yb1-xb29.google.com (mail-yb1-xb29.google.com [IPv6:2607:f8b0:4864:20::b29]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 493783592 for ; Thu, 13 Apr 2023 16:31:48 -0700 (PDT) Received: by mail-yb1-xb29.google.com with SMTP id v7so15722254ybi.0 for ; Thu, 13 Apr 2023 16:31:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ttaylorr-com.20221208.gappssmtp.com; s=20221208; t=1681428707; x=1684020707; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=+MSRfufCEBnlAzjUuqVX4v8aSYZAk11unkkEjgdcn4I=; b=X85E6aNOu6CyKx6KbjtXs3STNVt/CO3/IdN/AgKRHvNtViLU2xV1tdy8MZlx6o/vJE MgofWgK1aF2VvSqjxSNpUweoFw4bL6/eVKPaOi89Sn7eSzL5rqnaLAUXzKA4eArkBfcb +eIvo8gzeWnszEa8ZDOTVMt7ZSKhuxFookI3KrB6QhKjDoPl4k51Sb1xKuAoT+TdCtDX htOpLfxjV4C1tA4jkNjahsSSBBPn0ReGtGW3uEYxW9KxJESQ2d3KfAVTbsWrJ/8Gu+B7 twiHTiv7Rfy3csIBrFokAqOlT5o/1AgKlZNeDBeT2jiuhYc4kuNc9D1w5gd4IX63/k+k Lk0Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1681428707; x=1684020707; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=+MSRfufCEBnlAzjUuqVX4v8aSYZAk11unkkEjgdcn4I=; b=IL/xgkfj0bA5rFZD4BVtPTAmaTygPOU4U1IWV1AGVRHHRTYsBFgOgeNXdvIwQ7OVOW rKm0ayQR+jQJq3CWXwLJ/MvTiiYiBpR9v1cQz5wvdV957ksc2eFAsu5OjHwq6N86A6yf Fz8IuDx20DUeLdmgiBEBSxpQLATwHPxaveLIqiZOz+jsTdX8OwHR1cM9fyLaW0/8sMQe U8gJtw++sV/q6pZ1g3exMa1u1sfjckY7qCAmKkew5+Syh9vB8yXOKuw6GTWD907tSCG7 908Eg5I4rrTgmgTj1DDee531X9geOifPyrVyGX3yAENM5j1EvThESqa09EoMNlMt0gKM q2lw== X-Gm-Message-State: AAQBX9c4gvKL4gaHP6k0neKDUVaZJuLq8XglDFgc0JTQdiBIi3o/z8TO qD4vstehw1UHVxCv6xagw2eENtuSR4Zk9O5BUVbI1w== X-Google-Smtp-Source: AKy350Z1tvTznCKcDgL5324Mj1gaQUGZAVhZBxvGT9oCz95MOFNsiOODPmiljkgeFfhq6719/MEmFQ== X-Received: by 2002:a25:d751:0:b0:b67:663a:4c6e with SMTP id o78-20020a25d751000000b00b67663a4c6emr3803329ybg.25.1681428707249; Thu, 13 Apr 2023 16:31:47 -0700 (PDT) Received: from localhost (104-178-186-189.lightspeed.milwwi.sbcglobal.net. [104.178.186.189]) by smtp.gmail.com with ESMTPSA id 2-20020a250102000000b00b8f3b826e58sm779185ybb.19.2023.04.13.16.31.46 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 13 Apr 2023 16:31:47 -0700 (PDT) Date: Thu, 13 Apr 2023 19:31:46 -0400 From: Taylor Blau To: git@vger.kernel.org Cc: Jeff King , Junio C Hamano Subject: [PATCH 2/5] t/helper/test-hashmap.c: avoid using `strtok()` Message-ID: <0f199468a3b8375dbec0f56fdc831c3ac298eb4e.1681428696.git.me@ttaylorr.com> References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Avoid using the non-reentrant `strtok()` to separate the parts of each incoming command. Instead of replacing it with `strtok_r()`, let's instead use the more friendly `string_list_split_in_place_multi()`. Signed-off-by: Taylor Blau --- t/helper/test-hashmap.c | 30 +++++++++++++++++++++++------- 1 file changed, 23 insertions(+), 7 deletions(-) diff --git a/t/helper/test-hashmap.c b/t/helper/test-hashmap.c index 36ff07bd4be..902ceb55113 100644 --- a/t/helper/test-hashmap.c +++ b/t/helper/test-hashmap.c @@ -2,6 +2,7 @@ #include "git-compat-util.h" #include "hashmap.h" #include "strbuf.h" +#include "string-list.h" struct test_entry { @@ -150,6 +151,7 @@ static void perf_hashmap(unsigned int method, unsigned int rounds) */ int cmd__hashmap(int argc, const char **argv) { + struct string_list parts = STRING_LIST_INIT_NODUP; struct strbuf line = STRBUF_INIT; int icase; struct hashmap map = HASHMAP_INIT(test_entry_cmp, &icase); @@ -159,21 +161,34 @@ int cmd__hashmap(int argc, const char **argv) /* process commands from stdin */ while (strbuf_getline(&line, stdin) != EOF) { - char *cmd, *p1 = NULL, *p2 = NULL; + char *cmd, *p1, *p2; unsigned int hash = 0; struct test_entry *entry; + /* + * Because we memdup() the arguments out of the + * string_list before inserting them into the hashmap, + * it's OK to set its length back to zero to avoid + * re-allocating the items array once per line. + * + * By doing so, we'll instead overwrite the existing + * entries and avoid re-allocating. + */ + parts.nr = 0; /* break line into command and up to two parameters */ - cmd = strtok(line.buf, DELIM); + string_list_split_in_place_multi(&parts, line.buf, DELIM, 2); + /* ignore empty lines */ - if (!cmd || *cmd == '#') + if (!parts.nr) + continue; + if (!*parts.items[0].string || *parts.items[0].string == '#') continue; - p1 = strtok(NULL, DELIM); - if (p1) { + cmd = parts.items[0].string; + p1 = parts.nr >= 1 ? parts.items[1].string : NULL; + p2 = parts.nr >= 2 ? parts.items[2].string : NULL; + if (p1) hash = icase ? strihash(p1) : strhash(p1); - p2 = strtok(NULL, DELIM); - } if (!strcmp("add", cmd) && p1 && p2) { @@ -260,6 +275,7 @@ int cmd__hashmap(int argc, const char **argv) } } + string_list_clear(&parts, 0); strbuf_release(&line); hashmap_clear_and_free(&map, struct test_entry, ent); return 0; From patchwork Thu Apr 13 23:31:48 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Taylor Blau X-Patchwork-Id: 13210755 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1A632C77B6E for ; Thu, 13 Apr 2023 23:31:59 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229577AbjDMXb6 (ORCPT ); Thu, 13 Apr 2023 19:31:58 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58526 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229954AbjDMXb4 (ORCPT ); Thu, 13 Apr 2023 19:31:56 -0400 Received: from mail-yb1-xb31.google.com (mail-yb1-xb31.google.com [IPv6:2607:f8b0:4864:20::b31]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 0D5183592 for ; Thu, 13 Apr 2023 16:31:51 -0700 (PDT) Received: by mail-yb1-xb31.google.com with SMTP id by8so960712ybb.9 for ; Thu, 13 Apr 2023 16:31:51 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ttaylorr-com.20221208.gappssmtp.com; s=20221208; t=1681428710; x=1684020710; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=jo8JRwRzSH+SB1J4ZOUrIyRxeAmpW9jQ2+eORzP3B4c=; b=Zsxbw5GwsmdrnIBJ3JKq0j6rlEHdh0PrEmmIKASwGGMAtJtMwE3CMR3Wqq+zudI58W RDWTMmNNHX3NPx+vFw2q2+kuE7gw4CbkfSFEbCnMNMsRLdn2de8wZhscXge3GKaE2HbG ik6fy+1DrcgMoHBBBOnZB7RCP/B+IJ02AcKX775/NpFsc0r2MGbw4c3Wrlvt0ha3XC3t wvn0msE7Qm5pXAYNc/ajBcmPsHPrzyFfeN2ie796Vb4E6qOyyhMB2YdSQDarzLi2dAez DGtja1DjstMHVkse3/ZMfnB7C4mvhyQb5EL+kNwVaGTKhNKwmMvfi0LWFmKkEAi1Wnj5 AP5w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1681428710; x=1684020710; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=jo8JRwRzSH+SB1J4ZOUrIyRxeAmpW9jQ2+eORzP3B4c=; b=KEwoI4wM1ROwaoH6R9nFTvCwjwHCktWktVxkMayB8HD762+XlW6gxMoQX8nwW4DmzB i3mrAMT50dPYbhFt4HaxzfUbRdkwy2ogfDMh2WRnA5533ImkpO3E07U8ASPanqOThMud 8+3jx1TLe7saSozJrp8WQtAxokA//l+zmt0IZA65S9uT9gJ5I1FQLQVrYXuyBF8MtKfV WdoJwXhGyywV43s+xEzh0UlmF9F7xEjMsaALxtAiK9KOjFaZRcvX4HKmolQrhvI6B/fE 4BYMf+QIS18petE7OTWXDqzHCBWHEqovBRHqrCl0GhmSofPgXtLp0WAZWJ9lB6mGIOo9 3VMw== X-Gm-Message-State: AAQBX9eTyipSHRRUp6N49RTkq0GJwL4nx93bcbb+3Nn7/ZdOnDWOdG9/ KK78NPq8hoIivp/U4/sZJd3EG+FBQvVcE6RAWjPkxg== X-Google-Smtp-Source: AKy350aZcnCXAiPrYY0qwCAO//5buGM75UKLplTnId6cOdShv84nglUxZL0VBLotcWgIRR2qT9JxLg== X-Received: by 2002:a25:3743:0:b0:b8f:3eda:30b7 with SMTP id e64-20020a253743000000b00b8f3eda30b7mr4253612yba.13.1681428709982; Thu, 13 Apr 2023 16:31:49 -0700 (PDT) Received: from localhost (104-178-186-189.lightspeed.milwwi.sbcglobal.net. [104.178.186.189]) by smtp.gmail.com with ESMTPSA id y11-20020a25bb8b000000b00b7767ca747fsm763726ybg.28.2023.04.13.16.31.49 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 13 Apr 2023 16:31:49 -0700 (PDT) Date: Thu, 13 Apr 2023 19:31:48 -0400 From: Taylor Blau To: git@vger.kernel.org Cc: Jeff King , Junio C Hamano Subject: [PATCH 3/5] t/helper/test-oidmap.c: avoid using `strtok()` Message-ID: <135222d04e4d9cb85c149e0e12ea531d5f43d9cd.1681428696.git.me@ttaylorr.com> References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Apply similar treatment as in the previous commit to remove usage of `strtok()` from the "oidmap" test helper. Signed-off-by: Taylor Blau --- t/helper/test-oidmap.c | 20 ++++++++++++++------ 1 file changed, 14 insertions(+), 6 deletions(-) diff --git a/t/helper/test-oidmap.c b/t/helper/test-oidmap.c index a7b7b38df1f..cf4d2f7c085 100644 --- a/t/helper/test-oidmap.c +++ b/t/helper/test-oidmap.c @@ -4,6 +4,7 @@ #include "oidmap.h" #include "setup.h" #include "strbuf.h" +#include "string-list.h" /* key is an oid and value is a name (could be a refname for example) */ struct test_entry { @@ -25,6 +26,7 @@ struct test_entry { */ int cmd__oidmap(int argc UNUSED, const char **argv UNUSED) { + struct string_list parts = STRING_LIST_INIT_NODUP; struct strbuf line = STRBUF_INIT; struct oidmap map = OIDMAP_INIT; @@ -35,19 +37,24 @@ int cmd__oidmap(int argc UNUSED, const char **argv UNUSED) /* process commands from stdin */ while (strbuf_getline(&line, stdin) != EOF) { - char *cmd, *p1 = NULL, *p2 = NULL; + char *cmd, *p1, *p2; struct test_entry *entry; struct object_id oid; + /* see the comment in cmd__hashmap() */ + parts.nr = 0; /* break line into command and up to two parameters */ - cmd = strtok(line.buf, DELIM); + string_list_split_in_place_multi(&parts, line.buf, DELIM, 2); + /* ignore empty lines */ - if (!cmd || *cmd == '#') + if (!parts.nr) + continue; + if (!*parts.items[0].string || *parts.items[0].string == '#') continue; - p1 = strtok(NULL, DELIM); - if (p1) - p2 = strtok(NULL, DELIM); + cmd = parts.items[0].string; + p1 = parts.nr >= 1 ? parts.items[1].string : NULL; + p2 = parts.nr >= 2 ? parts.items[2].string : NULL; if (!strcmp("put", cmd) && p1 && p2) { @@ -108,6 +115,7 @@ int cmd__oidmap(int argc UNUSED, const char **argv UNUSED) } } + string_list_clear(&parts, 0); strbuf_release(&line); oidmap_free(&map, 1); return 0; From patchwork Thu Apr 13 23:31:51 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Taylor Blau X-Patchwork-Id: 13210756 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id F31D4C77B61 for ; Thu, 13 Apr 2023 23:32:02 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230281AbjDMXcB (ORCPT ); Thu, 13 Apr 2023 19:32:01 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58608 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229961AbjDMXb7 (ORCPT ); Thu, 13 Apr 2023 19:31:59 -0400 Received: from mail-yw1-x112f.google.com (mail-yw1-x112f.google.com [IPv6:2607:f8b0:4864:20::112f]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id CB4BD449F for ; Thu, 13 Apr 2023 16:31:53 -0700 (PDT) Received: by mail-yw1-x112f.google.com with SMTP id 00721157ae682-54fe3cd445aso7946207b3.5 for ; Thu, 13 Apr 2023 16:31:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ttaylorr-com.20221208.gappssmtp.com; s=20221208; t=1681428713; x=1684020713; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=m10wBKPY2t3mFe2nREbOAFi4XRN6si1/cU+jBNHsrh4=; b=Q8wIvZgtwwlvNK+5/gYT86XO3qgSQUClAsN6XGVf65BPhhnEWsvBbMD3bAyNdEJZNw rX2NC1D6fuRsF9vtS0WaQoURU3yk+Wpk+VTYxYA4oLnukzqTn6gN+xyUW4NpEqsZlMH+ xpYso7IhWNwigaV3DcMLtWyZftNTXuOmkaqNn8KubNTOJUsB5PrKIln3bXhhScilzxUl 5olFon6XUlSLdWP65mKnJG+QWcxsgPoXGY3F2CLdh9goNI3hy87zgb+8W3V4Q+ompECt LHPajPRkvHg+1sOLkdufljNU+t1m6ot+KuOpztv8iOc8AFD0OV41H96BxCkbjAFTUrPk 286w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1681428713; x=1684020713; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=m10wBKPY2t3mFe2nREbOAFi4XRN6si1/cU+jBNHsrh4=; b=gtx26VyvVXUR5Dq8ul5ApfpRzIERpobfy9D4XzSJT7DP4F4v+ro7p9ozprNHduxQJY 7zUUHxqpdCuJLN7yrJyjrPSFVuK/dDMYfTZuiEmx5M4hR8Nmp83jdub1e88hU5YPac2a TMEOjOKba76Stzq4gqCy09LT5tzXO1ATJ5gxw1pJ5tSRW8cop4NbgqNHgBUoQ/CRvHJe +3VMKBaNb6PMEVwP2F1wg1XezljsQBal2eWkL/yGrJ4wpX6BXUEPg41yoQagOxYCEMoU lrjku0/l4PcP6HvO+zbeUkHofcTu3HDi187poQY7gfnNHJz3XaDJihDhUWCOL0hXhhtL 8Vgg== X-Gm-Message-State: AAQBX9cqPC5Ea8IMXj6e4iSODM7jnNi8ugRQXWuHn1gIHcLXYqGGGjNc 9SDPE2QjKEp8etgrMFi7ApuuLfMEe2lbMfXAvaK26g== X-Google-Smtp-Source: AKy350at8tP8aC/spVbHHqeFhgUA6SoFrxxmuAA423CvpQz4WtFNUxuK8vkw6R+kIiHdQzteaM3PfA== X-Received: by 2002:a0d:e207:0:b0:541:6b00:f68d with SMTP id l7-20020a0de207000000b005416b00f68dmr3722523ywe.16.1681428712775; Thu, 13 Apr 2023 16:31:52 -0700 (PDT) Received: from localhost (104-178-186-189.lightspeed.milwwi.sbcglobal.net. [104.178.186.189]) by smtp.gmail.com with ESMTPSA id p125-20020a819883000000b00545a0818490sm828170ywg.32.2023.04.13.16.31.52 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 13 Apr 2023 16:31:52 -0700 (PDT) Date: Thu, 13 Apr 2023 19:31:51 -0400 From: Taylor Blau To: git@vger.kernel.org Cc: Jeff King , Junio C Hamano Subject: [PATCH 4/5] t/helper/test-json-writer.c: avoid using `strtok()` Message-ID: References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Apply similar treatment as in the previous commit to remove usage of `strtok()` from the "oidmap" test helper. Each of the different commands that the "json-writer" helper accepts pops the next space-delimited token from the current line and interprets it as a string, integer, or double (with the exception of the very first token, which is the command itself). To accommodate this, split the line in place by the space character, and pass the corresponding string_list to each of the specialized `get_s()`, `get_i()`, and `get_d()` functions. `get_i()` and `get_d()` are thin wrappers around `get_s()` that convert their result into the appropriate type by either calling `strtol()` or `strtod()`, respectively. In `get_s()`, we mark the token as "consumed" by incrementing the `consumed_nr` counter, indicating how many tokens we have read up to that point. Because each of these functions needs the string-list parts, the number of tokens consumed, and the line number, these three are wrapped up in to a struct representing the line state. Signed-off-by: Taylor Blau --- t/helper/test-json-writer.c | 76 +++++++++++++++++++++++-------------- 1 file changed, 48 insertions(+), 28 deletions(-) diff --git a/t/helper/test-json-writer.c b/t/helper/test-json-writer.c index 86887f53203..64bfa1c5f3c 100644 --- a/t/helper/test-json-writer.c +++ b/t/helper/test-json-writer.c @@ -1,5 +1,6 @@ #include "test-tool.h" #include "json-writer.h" +#include "string-list.h" static const char *expect_obj1 = "{\"a\":\"abc\",\"b\":42,\"c\":true}"; static const char *expect_obj2 = "{\"a\":-1,\"b\":2147483647,\"c\":0}"; @@ -394,35 +395,41 @@ static int unit_tests(void) return 0; } -static void get_s(int line_nr, char **s_in) +struct line { + struct string_list *parts; + size_t consumed_nr; + int nr; +}; + +static void get_s(struct line *line, char **s_in) { - *s_in = strtok(NULL, " "); - if (!*s_in) - die("line[%d]: expected: ", line_nr); + if (line->consumed_nr > line->parts->nr) + die("line[%d]: expected: ", line->nr); + *s_in = line->parts->items[line->consumed_nr++].string; } -static void get_i(int line_nr, intmax_t *s_in) +static void get_i(struct line *line, intmax_t *s_in) { char *s; char *endptr; - get_s(line_nr, &s); + get_s(line, &s); *s_in = strtol(s, &endptr, 10); if (*endptr || errno == ERANGE) - die("line[%d]: invalid integer value", line_nr); + die("line[%d]: invalid integer value", line->nr); } -static void get_d(int line_nr, double *s_in) +static void get_d(struct line *line, double *s_in) { char *s; char *endptr; - get_s(line_nr, &s); + get_s(line, &s); *s_in = strtod(s, &endptr); if (*endptr || errno == ERANGE) - die("line[%d]: invalid float value", line_nr); + die("line[%d]: invalid float value", line->nr); } static int pretty; @@ -453,6 +460,7 @@ static char *get_trimmed_line(char *buf, int buf_size) static int scripted(void) { + struct string_list parts = STRING_LIST_INIT_NODUP; struct json_writer jw = JSON_WRITER_INIT; char buf[MAX_LINE_LENGTH]; char *line; @@ -470,66 +478,77 @@ static int scripted(void) die("expected first line to be 'object' or 'array'"); while ((line = get_trimmed_line(buf, MAX_LINE_LENGTH)) != NULL) { + struct line state = { 0 }; char *verb; char *key; char *s_value; intmax_t i_value; double d_value; - line_nr++; + state.parts = &parts; + state.nr = ++line_nr; - verb = strtok(line, " "); + /* see the comment in cmd__hashmap() */ + parts.nr = 0; + /* break line into command and zero or more tokens */ + string_list_split_in_place(&parts, line, ' ', -1); + + /* ignore empty lines */ + if (!parts.nr || !*parts.items[0].string) + continue; + + verb = parts.items[state.consumed_nr++].string; if (!strcmp(verb, "end")) { jw_end(&jw); } else if (!strcmp(verb, "object-string")) { - get_s(line_nr, &key); - get_s(line_nr, &s_value); + get_s(&state, &key); + get_s(&state, &s_value); jw_object_string(&jw, key, s_value); } else if (!strcmp(verb, "object-int")) { - get_s(line_nr, &key); - get_i(line_nr, &i_value); + get_s(&state, &key); + get_i(&state, &i_value); jw_object_intmax(&jw, key, i_value); } else if (!strcmp(verb, "object-double")) { - get_s(line_nr, &key); - get_i(line_nr, &i_value); - get_d(line_nr, &d_value); + get_s(&state, &key); + get_i(&state, &i_value); + get_d(&state, &d_value); jw_object_double(&jw, key, i_value, d_value); } else if (!strcmp(verb, "object-true")) { - get_s(line_nr, &key); + get_s(&state, &key); jw_object_true(&jw, key); } else if (!strcmp(verb, "object-false")) { - get_s(line_nr, &key); + get_s(&state, &key); jw_object_false(&jw, key); } else if (!strcmp(verb, "object-null")) { - get_s(line_nr, &key); + get_s(&state, &key); jw_object_null(&jw, key); } else if (!strcmp(verb, "object-object")) { - get_s(line_nr, &key); + get_s(&state, &key); jw_object_inline_begin_object(&jw, key); } else if (!strcmp(verb, "object-array")) { - get_s(line_nr, &key); + get_s(&state, &key); jw_object_inline_begin_array(&jw, key); } else if (!strcmp(verb, "array-string")) { - get_s(line_nr, &s_value); + get_s(&state, &s_value); jw_array_string(&jw, s_value); } else if (!strcmp(verb, "array-int")) { - get_i(line_nr, &i_value); + get_i(&state, &i_value); jw_array_intmax(&jw, i_value); } else if (!strcmp(verb, "array-double")) { - get_i(line_nr, &i_value); - get_d(line_nr, &d_value); + get_i(&state, &i_value); + get_d(&state, &d_value); jw_array_double(&jw, i_value, d_value); } else if (!strcmp(verb, "array-true")) @@ -552,6 +571,7 @@ static int scripted(void) printf("%s\n", jw.json.buf); jw_release(&jw); + string_list_clear(&parts, 0); return 0; } From patchwork Thu Apr 13 23:31:54 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Taylor Blau X-Patchwork-Id: 13210757 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 12031C77B61 for ; Thu, 13 Apr 2023 23:32:09 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230517AbjDMXcH (ORCPT ); Thu, 13 Apr 2023 19:32:07 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58598 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230184AbjDMXcC (ORCPT ); Thu, 13 Apr 2023 19:32:02 -0400 Received: from mail-yw1-x1129.google.com (mail-yw1-x1129.google.com [IPv6:2607:f8b0:4864:20::1129]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 6E804422F for ; Thu, 13 Apr 2023 16:31:56 -0700 (PDT) Received: by mail-yw1-x1129.google.com with SMTP id 00721157ae682-54fe3cd445aso7947647b3.5 for ; Thu, 13 Apr 2023 16:31:56 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ttaylorr-com.20221208.gappssmtp.com; s=20221208; t=1681428715; x=1684020715; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=8DzunMpXbpcg9jv9QPgZ6Aum2+UnYUBJ80s31+jU5nc=; b=zAyw4ubwUM/d5G9vAizOc8+qHkuNCPnSawz+LfiYYqLRdVUSDI/92wojMvXq/PMVeu tMYUO2xGMgQiZa0u4NBD4Y/jGIO46ALfdBwgLRQS62E4vUXcRlLbYpERPIEHb4usWRsP T9QtfdiQfX76duqb1ZVhZL5LXX/D0CQxrP0G5pLJvQMAJp03LGmiVkiLpJYsebt/KNXl 9b5v2phYk+uTXAB5tmxdKG/1sXFKEa5px/k/2LC137cvBJHu/GNxj6oAIMZBV0FNQStn xkW/TsOUiw1N03hFrO49go+P5jYoE3drCTCCzOmvPJ1kXTtvAFTJ/p+JsoOXF48fERw5 b+zA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1681428715; x=1684020715; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=8DzunMpXbpcg9jv9QPgZ6Aum2+UnYUBJ80s31+jU5nc=; b=JjsydE/g0/fUr0pxJ6mssVY04X9Zn2xaPYgRgl1p6XJrAVE3iCibmKZJz8v/fbfE9D FR9Splr4zc0Dan0BU1NVegsmE7VG6/lDDqZMdkaO7anm8BqKc7EzndUGmBpuuF8T/wgt KtOZaOeJHYjh/HPk8NtLBGjL6bZClqNK3CjY5gVR1cC03jNscELOn56UnhCjlzaGRa9e qKgVx2pP1weMpDl/ffWYUHzwXcdK6Vhzj7XMAPyR+R1mayuyAVxHfnQh7jSwf7ofUmpZ 9v1+AfyLiR9/9Ahk9ZtNTAQcKBsbzPmnRTzpKhFXudJMxCtcdKwl0hpJc/0vLFohapQa 41HQ== X-Gm-Message-State: AAQBX9euZjQmPqsLKT5s3UrBshllU/hflFjKHHxABU+sc8O+RPNwmcv7 KXmvhqMoeXM4R2ZEyWXaQe9eRExFGVUuiH2vYuFp/g== X-Google-Smtp-Source: AKy350Ykh7rbPSJUQXa9/WOlbcrx0YI5Ys/HPVZagfaF01JPM7KACYr3TBdWbKnkA2pLpbr2Lyl6Wg== X-Received: by 2002:a81:6ac3:0:b0:54f:1185:b57f with SMTP id f186-20020a816ac3000000b0054f1185b57fmr3850201ywc.29.1681428715420; Thu, 13 Apr 2023 16:31:55 -0700 (PDT) Received: from localhost (104-178-186-189.lightspeed.milwwi.sbcglobal.net. [104.178.186.189]) by smtp.gmail.com with ESMTPSA id a6-20020a814d06000000b0054ee8426468sm810346ywb.21.2023.04.13.16.31.55 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 13 Apr 2023 16:31:55 -0700 (PDT) Date: Thu, 13 Apr 2023 19:31:54 -0400 From: Taylor Blau To: git@vger.kernel.org Cc: Jeff King , Junio C Hamano Subject: [PATCH 5/5] banned.h: mark `strtok()`, `strtok_r()` as banned Message-ID: <1d955f8bc6d2797def516897d019a186e461b648.1681428696.git.me@ttaylorr.com> References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org `strtok_r()` is reentrant, but `strtok()` is not, meaning that using it is not thread-safe. We could ban `strtok()` and force callers to use its reentrant counterpart, but there are a few drawbacks to doing so: - `strtok_r()` forces the caller to maintain an extra string pointer to pass as its `saveptr` value - `strtok_r()` also requires that its `saveptr` value be unmodified between calls. - `strtok()` (and by extension, `strtok_r()`) is confusing when used across multiple functions, since the caller is supposed to pass NULL as its first argument after the first call. This makes it difficult to determine what string is actually being tokenized without clear dataflow. So while we could ban only `strtok()`, there really is no good reason to use either when callers could instead use the much friendlier `string_list_split_in_place()` API, which avoids the above issues. Signed-off-by: Taylor Blau --- banned.h | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/banned.h b/banned.h index 6ccf46bc197..9bd23ce5732 100644 --- a/banned.h +++ b/banned.h @@ -18,6 +18,12 @@ #define strncpy(x,y,n) BANNED(strncpy) #undef strncat #define strncat(x,y,n) BANNED(strncat) +#if 0 +#undef strtok +#define strtok(x,y) BANNED(strtok) +#undef strtok_r +#define strtok_r(x,y,z) BANNED(strtok_r) +#endif #undef sprintf #undef vsprintf