| Message ID | d15ed626de65c51ef2ba31020eeb2111fb8e091f.1596675905.git.gitgitgadget@gmail.com (mailing list archive) |
|---|---|
| State | Superseded |
| Headers | show |
| Series | Fit the building tools to Plan 9 environment | expand |
On 2020-08-06 at 01:05:03, lufia via GitGitGadget wrote: > From: lufia <lufia@lufia.org> > > That haven't any commands: cut, expr and printf. Is this ANSI/POSIX environment the one mentioned at [0]? That page describes it as supporting POSIX 1003.1-1990, which is a bit dated. I think we generally assume one has the 2001 edition or later, so you'll have your work cut out for you. > And its sed(1)'s label is limited to maximum seven characters. > Therefore I replaced some labels to drop a character. > > * close -> cl > * continue -> cont (cnt is used for count) > * line -> ln > * hered -> hdoc > * shell -> sh > * string -> str > > Signed-off-by: lufia <lufia@lufia.org> I will note that usually the project prefers to have a human's personal name here and in the commit metadata instead of a username. Junio may chime in here with an opinion. > command_list () { > - eval "grep -ve '^#' $exclude_programs" <"$1" > + eval "grep -v -e '^#' $exclude_programs" <"$1" Is it really the case that Plan 9's grep cannot deal with bundled short options? That seems to be a significant departure from POSIX and Unix behavior. Regardless, this should be explained in the commit message. > get_categories () { > - tr ' ' '\n'| > + tr ' ' '\012'| Okay, I guess. Is this something we need to handle elsewhere as well? The commit message should tell us why this is necessary, and what Plan 9 does and doesn't support. > grep -v '^$' | > sort | > uniq > @@ -18,13 +18,13 @@ get_categories () { > > category_list () { > command_list "$1" | > - cut -c 40- | > + awk '{ print substr($0, 40) }' | I can tell that you haven't gotten the test suite working because I've added a large number of cut invocations there. I suspect you're going to need to provide a portability wrapper there that implements it using awk, sed, or perl. > +if test -z "$(echo -n)" > +then > + alias print='echo -n' > +else > + alias print='printf %s' > +fi Let's avoid an alias here (especially with a common builtin name) and instead use a shell function. Maybe like this (not tab-indented): print_nonl () { if command -v printf >/dev/null 2>&1 then printf "%s" "$@" else echo -n "$@" fi } Notice also that we prefer the standard form and fall back to the nonstandard form if the system is less capable. I don't know if Plan 9 supports "command -v"; "type" may be preferable, but isn't supported by some other shells (e.g., posh). For portability reasons, we may need to try to run printf and see if it fails. This is also going to need some patching in the testsuite, since we use printf extensively (more than 1300 times). I do hope you have perl available. [0] http://doc.cat-v.org/plan_9/4th_edition/papers/ape
> On 2020-08-06 at 01:05:03, lufia via GitGitGadget wrote: > > From: lufia <lufia@lufia.org> > > > > That haven't any commands: cut, expr and printf. > > Is this ANSI/POSIX environment the one mentioned at [0]? That page > describes it as supporting POSIX 1003.1-1990, which is a bit dated. I > think we generally assume one has the 2001 edition or later, so you'll > have your work cut out for you. Yes, the layer I told is APE. I guess originally APE might be introduced for porting Ghostscript to Plan 9. > > And its sed(1)'s label is limited to maximum seven characters. > > Therefore I replaced some labels to drop a character. > > > > * close -> cl > > * continue -> cont (cnt is used for count) > > * line -> ln > > * hered -> hdoc > > * shell -> sh > > * string -> str > > > > Signed-off-by: lufia <lufia@lufia.org> > > I will note that usually the project prefers to have a human's personal > name here and in the commit metadata instead of a username. Junio may > chime in here with an opinion. I see. I will rename them. > > command_list () { > > - eval "grep -ve '^#' $exclude_programs" <"$1" > > + eval "grep -v -e '^#' $exclude_programs" <"$1" > > Is it really the case that Plan 9's grep cannot deal with bundled short > options? That seems to be a significant departure from POSIX and Unix > behavior. Regardless, this should be explained in the commit message. This is awful. But now, APE's grep (/bin/ape/grep) is a simple wrapper for native grep (/bin/grep), its option parser is a very rough implementation. https://github.com/0intro/plan9-contrib/blob/master/rc/bin/ape/grep > > get_categories () { > > - tr ' ' '\n'| > > + tr ' ' '\012'| > > Okay, I guess. Is this something we need to handle elsewhere as well? > The commit message should tell us why this is necessary, and what Plan 9 > does and doesn't support. Yeah. I will edit the message. Plan 9's tr(1) handles only \(16 bit octal) and \x(16 bit hexadecimal) escape sequences. If another character after leading backslash, tr(1) will replace \c to c. > > grep -v '^$' | > > sort | > > uniq > > @@ -18,13 +18,13 @@ get_categories () { > > > > category_list () { > > command_list "$1" | > > - cut -c 40- | > > + awk '{ print substr($0, 40) }' | > > I can tell that you haven't gotten the test suite working because I've > added a large number of cut invocations there. I suspect you're going > to need to provide a portability wrapper there that implements it using > awk, sed, or perl. I see. If I'd like to put those wrappers to the repository, is there the best place for them? > > +if test -z "$(echo -n)" > > +then > > + alias print='echo -n' > > +else > > + alias print='printf %s' > > +fi > > Let's avoid an alias here (especially with a common builtin name) and > instead use a shell function. Maybe like this (not tab-indented): > > print_nonl () { > if command -v printf >/dev/null 2>&1 > then > printf "%s" "$@" > else > echo -n "$@" > fi > } > > Notice also that we prefer the standard form and fall back to the > nonstandard form if the system is less capable. I don't know if Plan 9 > supports "command -v"; "type" may be preferable, but isn't supported by > some other shells (e.g., posh). For portability reasons, we may need to > try to run printf and see if it fails. > > This is also going to need some patching in the testsuite, since we use > printf extensively (more than 1300 times). I do hope you have perl > available. In fact, Plan 9's ape/sh is pdksh, so it supports "command -v". However I think, like the above comment, it might be better to create the printf(1) wrapper. --- kadota > [0] http://doc.cat-v.org/plan_9/4th_edition/papers/ape > -- > brian m. carlson: Houston, Texas, US
"brian m. carlson" <sandals@crustytoothpaste.net> writes: > I will note that usually the project prefers to have a human's personal > name here and in the commit metadata instead of a username. Junio may > chime in here with an opinion. Thanks, I just did. >> command_list () { >> - eval "grep -ve '^#' $exclude_programs" <"$1" >> + eval "grep -v -e '^#' $exclude_programs" <"$1" > > Is it really the case that Plan 9's grep cannot deal with bundled short > options? That seems to be a significant departure from POSIX and Unix > behavior. Regardless, this should be explained in the commit message. I am not interested in this ball of wax, but there are some pieces that are worth salvaging. This is one of those good bits. diff --git a/GIT-VERSION-GEN b/GIT-VERSION-GEN index 9db2f4feab..a7cc01caf9 100755 --- a/GIT-VERSION-GEN +++ b/GIT-VERSION-GEN @@ -26,7 +26,7 @@ else VN="$DEF_VER" fi -VN=$(expr "$VN" : v*'\(.*\)') +VN=${VN#v} if test -r $GVF then BUT. Dealing with "grep" that cannot take "-ve" and forces developers to spell it as "-v -e" is not one of them. So is forbidding use of "cut". >> get_categories () { >> - tr ' ' '\n'| >> + tr ' ' '\012'| > > Okay, I guess. Is this something we need to handle elsewhere as well? > The commit message should tell us why this is necessary, and what Plan 9 > does and doesn't support. Yeah, POSIX does want you to understand '\n' and others, but this is within reason for portability that I think we could swallow. >> +if test -z "$(echo -n)" >> +then >> + alias print='echo -n' >> +else >> + alias print='printf %s' >> +fi > > Let's avoid an alias here (especially with a common builtin name) and > instead use a shell function. Maybe like this (not tab-indented): > > print_nonl () { > if command -v printf >/dev/null 2>&1 > then > printf "%s" "$@" > else > echo -n "$@" > fi > } I'd rather not to see this done; "echo -n" and "echo '...\c'" are not portable and we do want people to use 'printf'. This directly goes against it. > This is also going to need some patching in the testsuite, since we use > printf extensively (more than 1300 times). I do hope you have perl > available. Eh, so what would Plan9 people do with Perl? Write a single-liner 'printf' script and put it in a directory on their $PATH? I am not sure if it is worth crippling the toolset our developers are allowed to choose from and use in our scripts and tests like this patch tries to do. If this were Windows, it might have been a different story, but what we are talking about is Plan 9, which had the last "fourth edition" in 2002, right? During the 18 years since then, none of its users and developers work on porting many OSS packages, whose primary user base are on POSIXy systems, to it and we need to apply patches like these to each and every OSS packages of interest? I would have expected that any exotic-minority-but-thriving-platform would be able to tell its users "here are ports of POSIXy programs---install them and it can become usable by those who only know Linux"? So, I dunno.
On 2020-08-06 at 13:49:43, Kyohei Kadota wrote: > I see. If I'd like to put those wrappers to the repository, is there > the best place for them? If you can stuff them in a shell function, then I'd just put those in a file like t/test-lib-wrappers.sh and conditionally source them from t/test-lib.sh. Otherwise, I'd create a subdirectory of t and then add that to the PATH if necessary. If you're going to use it outside of the testsuite, then maybe compat/scripts might be a good idea. > In fact, Plan 9's ape/sh is pdksh, so it supports "command -v". > However I think, like the above comment, it might be better to create > the printf(1) wrapper. Awesome. I haven't used pdksh in a long time, but if it supports that, all the better. I'm surprised it doesn't have printf as a builtin, though.
On Thu, Aug 6, 2020 at 7:52 PM brian m. carlson <sandals@crustytoothpaste.net> wrote: > On 2020-08-06 at 13:49:43, Kyohei Kadota wrote: > > I see. If I'd like to put those wrappers to the repository, is there > > the best place for them? > > If you can stuff them in a shell function, then I'd just put those in a > file like t/test-lib-wrappers.sh and conditionally source them from > t/test-lib.sh. There are already platform-specific shell functions in test-lib.sh[1] and some larger compatibility functions in test-lib-functions.sh[2], so those might be better locations for Plan9 compatibility wrappers than creating a new file (or not?). [1]: see the section commented: "Fix some commands on Windows, and other OS-specific things" [2]: see, for instance, the mingw_test_cmp() function
> > "brian m. carlson" <sandals@crustytoothpaste.net> writes: > > > I will note that usually the project prefers to have a human's personal > > name here and in the commit metadata instead of a username. Junio may > > chime in here with an opinion. > > Thanks, I just did. > > >> command_list () { > >> - eval "grep -ve '^#' $exclude_programs" <"$1" > >> + eval "grep -v -e '^#' $exclude_programs" <"$1" > > > > Is it really the case that Plan 9's grep cannot deal with bundled short > > options? That seems to be a significant departure from POSIX and Unix > > behavior. Regardless, this should be explained in the commit message. > > I am not interested in this ball of wax, but there are some pieces > that are worth salvaging. This is one of those good bits. > > diff --git a/GIT-VERSION-GEN b/GIT-VERSION-GEN > index 9db2f4feab..a7cc01caf9 100755 > --- a/GIT-VERSION-GEN > +++ b/GIT-VERSION-GEN > @@ -26,7 +26,7 @@ else > VN="$DEF_VER" > fi > > -VN=$(expr "$VN" : v*'\(.*\)') > +VN=${VN#v} > > if test -r $GVF > then > > BUT. > > Dealing with "grep" that cannot take "-ve" and forces developers to > spell it as "-v -e" is not one of them. So is forbidding use of > "cut". I see. I will rewrite Plan 9's ape/grep to handle bundled options well. > >> get_categories () { > >> - tr ' ' '\n'| > >> + tr ' ' '\012'| > > > > Okay, I guess. Is this something we need to handle elsewhere as well? > > The commit message should tell us why this is necessary, and what Plan 9 > > does and doesn't support. > > Yeah, POSIX does want you to understand '\n' and others, but this is > within reason for portability that I think we could swallow. I'm happy, thanks. > >> +if test -z "$(echo -n)" > >> +then > >> + alias print='echo -n' > >> +else > >> + alias print='printf %s' > >> +fi > > > > Let's avoid an alias here (especially with a common builtin name) and > > instead use a shell function. Maybe like this (not tab-indented): > > > > print_nonl () { > > if command -v printf >/dev/null 2>&1 > > then > > printf "%s" "$@" > > else > > echo -n "$@" > > fi > > } > > I'd rather not to see this done; "echo -n" and "echo '...\c'" are > not portable and we do want people to use 'printf'. This directly > goes against it. > > > This is also going to need some patching in the testsuite, since we use > > printf extensively (more than 1300 times). I do hope you have perl > > available. > > Eh, so what would Plan9 people do with Perl? Write a single-liner > 'printf' script and put it in a directory on their $PATH? > > I am not sure if it is worth crippling the toolset our developers > are allowed to choose from and use in our scripts and tests like > this patch tries to do. Like other topics, I decide to implement printf(1). These new POSIX tools will be installed to Plan 9 systems from other repo or patches before building git. > If this were Windows, it might have been a different story, but what > we are talking about is Plan 9, which had the last "fourth edition" > in 2002, right? During the 18 years since then, none of its users > and developers work on porting many OSS packages, whose primary user > base are on POSIXy systems, to it and we need to apply patches like > these to each and every OSS packages of interest? I would have > expected that any exotic-minority-but-thriving-platform would be > able to tell its users "here are ports of POSIXy programs---install > them and it can become usable by those who only know Linux"? > > So, I dunno. Final official distribution from Bell labs was released in Jan 2015. Now, There are some distributions that forked from official Plan 9 and are maintained by that community, such as 9legacy, 9front or others. I will try to implement POSIX toolsets if it is needed because POSIX is a large spec. Thanks for your advice.
diff --git a/GIT-VERSION-GEN b/GIT-VERSION-GEN index 9db2f4feab..a7cc01caf9 100755 --- a/GIT-VERSION-GEN +++ b/GIT-VERSION-GEN @@ -26,7 +26,7 @@ else VN="$DEF_VER" fi -VN=$(expr "$VN" : v*'\(.*\)') +VN=${VN#v} if test -r $GVF then diff --git a/generate-cmdlist.sh b/generate-cmdlist.sh index 45fecf8bdf..8344ca6264 100755 --- a/generate-cmdlist.sh +++ b/generate-cmdlist.sh @@ -6,11 +6,11 @@ die () { } command_list () { - eval "grep -ve '^#' $exclude_programs" <"$1" + eval "grep -v -e '^#' $exclude_programs" <"$1" } get_categories () { - tr ' ' '\n'| + tr ' ' '\012'| grep -v '^$' | sort | uniq @@ -18,13 +18,13 @@ get_categories () { category_list () { command_list "$1" | - cut -c 40- | + awk '{ print substr($0, 40) }' | get_categories } get_synopsis () { sed -n ' - /^NAME/,/'"$1"'/H + /^NAME/,/'"$1"'/h ${ x s/.*'"$1"' - \(.*\)/N_("\1")/ @@ -60,16 +60,23 @@ define_category_names () { echo "};" } +if test -z "$(echo -n)" +then + alias print='echo -n' +else + alias print='printf %s' +fi + print_command_list () { echo "static struct cmdname_help command_list[] = {" command_list "$1" | while read cmd rest do - printf " { \"$cmd\", $(get_synopsis $cmd), 0" + print " { \"$cmd\", $(get_synopsis $cmd), 0" for cat in $(echo "$rest" | get_categories) do - printf " | CAT_$cat" + print " | CAT_$cat" done echo " }," done diff --git a/t/chainlint.sed b/t/chainlint.sed index 70df40e34b..8a25c5b855 100644 --- a/t/chainlint.sed +++ b/t/chainlint.sed @@ -117,7 +117,7 @@ /^[ ]*!*[ ]*(..*)[ ]*[0-9]*[<>|&]/boneline # multi-line "(...\n...)" -/^[ ]*(/bsubshell +/^[ ]*(/bsubsh # innocuous line -- print it and advance to next line b @@ -130,11 +130,11 @@ b } b -:subshell +:subsh # bare "(" line? -- stash for later printing /^[ ]*([ ]*$/ { h - bnextline + bnextln } # "(..." line -- split off and stash "(", then process "..." as its own line x @@ -143,7 +143,7 @@ x s/(// bslurp -:nextline +:nextln N s/.*\n// @@ -151,10 +151,10 @@ s/.*\n// # incomplete line "...\" /\\$/bicmplte # multi-line quoted string "...\n..."? -/"/bdqstring +/"/bdqstr # multi-line quoted string '...\n...'? (but not contraction in string "it's") /'/{ - /"[^'"]*'[^'"]*"/!bsqstring + /"[^'"]*'[^'"]*"/!bsqstr } :folded # here-doc -- swallow it @@ -163,8 +163,8 @@ s/.*\n// # before closing ")", "done", "elsif", "else", or "fi" will need to be # re-visited to drop "suspect" marking since final line of those constructs # legitimately lacks "&&", so "suspect" mark must be removed -/^[ ]*#/bnextline -/^[ ]*$/bnextline +/^[ ]*#/bnextln +/^[ ]*$/bnextln # in-line comment -- strip it (but not "#" in a string, Bash ${#...} array # length, or Perforce "//depot/path#42" revision in filespec) /[ ]#/{ @@ -175,22 +175,22 @@ s/.*\n// # multi-line "case ... esac" /^[ ]*case[ ]..*[ ]in/bcase # multi-line "for ... done" or "while ... done" -/^[ ]*for[ ]..*[ ]in/bcontinue -/^[ ]*while[ ]/bcontinue -/^[ ]*do[ ]/bcontinue -/^[ ]*do[ ]*$/bcontinue -/;[ ]*do/bcontinue +/^[ ]*for[ ]..*[ ]in/bcont +/^[ ]*while[ ]/bcont +/^[ ]*do[ ]/bcont +/^[ ]*do[ ]*$/bcont +/;[ ]*do/bcont /^[ ]*done[ ]*&&[ ]*$/bdone /^[ ]*done[ ]*$/bdone /^[ ]*done[ ]*[<>|]/bdone /^[ ]*done[ ]*)/bdone -/||[ ]*exit[ ]/bcontinue -/||[ ]*exit[ ]*$/bcontinue +/||[ ]*exit[ ]/bcont +/||[ ]*exit[ ]*$/bcont # multi-line "if...elsif...else...fi" -/^[ ]*if[ ]/bcontinue -/^[ ]*then[ ]/bcontinue -/^[ ]*then[ ]*$/bcontinue -/;[ ]*then/bcontinue +/^[ ]*if[ ]/bcont +/^[ ]*then[ ]/bcont +/^[ ]*then[ ]*$/bcont +/;[ ]*then/bcont /^[ ]*elif[ ]/belse /^[ ]*elif[ ]*$/belse /^[ ]*else[ ]/belse @@ -234,10 +234,10 @@ s/.*\n// } } # line ends with pipe "...|" -- valid; not missing "&&" -/|[ ]*$/bcontinue +/|[ ]*$/bcont # missing end-of-line "&&" -- mark suspect /&&[ ]*$/!s/^/?!AMP?!/ -:continue +:cont # retrieve and print previous line x n @@ -250,7 +250,7 @@ s/\\\n// bslurp # check for multi-line double-quoted string "...\n..." -- fold to one line -:dqstring +:dqstr # remove all quote pairs s/"\([^"]*\)"/@!\1@!/g # done if no dangling quote @@ -258,13 +258,13 @@ s/"\([^"]*\)"/@!\1@!/g # otherwise, slurp next line and try again N s/\n// -bdqstring +bdqstr :dqdone s/@!/"/g bfolded # check for multi-line single-quoted string '...\n...' -- fold to one line -:sqstring +:sqstr # remove all quote pairs s/'\([^']*\)'/@!\1@!/g # done if no dangling quote @@ -272,7 +272,7 @@ s/'\([^']*\)'/@!\1@!/g # otherwise, slurp next line and try again N s/\n// -bsqstring +bsqstr :sqdone s/@!/'/g bfolded @@ -282,11 +282,11 @@ bfolded :heredoc s/^\(.*\)<<[ ]*[-\\'"]*\([A-Za-z0-9_][A-Za-z0-9_]*\)['"]*/<\2>\1<</ s/[ ]*<<// -:heredsub +:hdocsub N /^<\([^>]*\)>.*\n[ ]*\1[ ]*$/!{ s/\n.*$// - bheredsub + bhdocsub } s/^<[^>]*>// s/\n.*$// @@ -305,7 +305,7 @@ bcase x s/?!AMP?!// x -bcontinue +bcont # found "done" closing for-loop or while-loop, or "fi" closing if-then -- drop # "suspect" from final contained line since that line legitimately lacks "&&" @@ -321,10 +321,10 @@ bchkchn # found nested multi-line "(...\n...)" -- pass through untouched :nest x -:nstslurp +:nstslrp n # closing ")" on own line -- stop nested slurp -/^[ ]*)/bnstclose +/^[ ]*)/bnstcl # comment -- not closing ")" if in comment /^[ ]*#/bnstcnt # "$((...))" -- arithmetic expansion; not closing ")" @@ -332,11 +332,11 @@ n # "$(...)" -- command substitution; not closing ")" /\$([^)][^)]*)[^)]*$/bnstcnt # closing "...)" -- stop nested slurp -/)/bnstclose +/)/bnstcl :nstcnt x -bnstslurp -:nstclose +bnstslrp +:nstcl s/^/>>/ # is it "))" which closes nested and parent subshells? /)[ ]*)/bslurp