| Message ID | xmqqsg9uxchb.fsf@gitster.c.googlers.com (mailing list archive) |
|---|---|
| State | Accepted |
| Commit | 8cfbe66736ee0df7747b45c1d4b59c3fb9ab22ee |
| Headers | show |
| Series | Makefile: enable -Wsparse-error for DEVELOPER build | expand |
Hi Junio, On 31/10/2020 22:22, Junio C Hamano wrote: > With -Wsparse-error, "make sparse" would fail, instead of just > giving a warning message. > > Signed-off-by: Junio C Hamano <gitster@pobox.com> > --- > config.mak.dev | 1 + > 1 file changed, 1 insertion(+) > > diff --git c/config.mak.dev w/config.mak.dev > index 3126a5364d..022fb58218 100644 > --- c/config.mak.dev > +++ w/config.mak.dev > @@ -1,5 +1,6 @@ > ifeq ($(filter no-error,$(DEVOPTS)),) > DEVELOPER_CFLAGS += -Werror > +SPARSE_FLAGS += -Wsparse-error > endif > ifneq ($(filter pedantic,$(DEVOPTS)),) > DEVELOPER_CFLAGS += -pedantic > I certainly wouldn't object to such a patch, but I'm not sure who it would actually help. ;-) As you may already know, I 'make sparse >sp-out 2>&1' on the master branch, use vim to check for warnings/errors, and then diff the corresponding files for 'next' and 'seen' branches (nsp-out, ssp-out). Similarly, I 'make -k hdr-check >hcout 2>&1' on the master branch, use vim ... etc. Note, however, the use of '-k' on the make invocation to make sure I catch all warnings/errors, since '-Werror' is given with the DEVELOPER variable set. So, I would have to do the same with the 'make sparse' invocation after this patch [1]. Yes, anybody who just does 'make sparse' will notice the failure, so that would be a definite improvement. (How many people run 'make sparse' though?). It probably would help anybody using 'cgcc' to develop, but this doesn't quite work (I just tried this for the first time in ages and was surprised to find it almost works for me, YMMV): $ git branch -v | grep '^\*' * seen 4141ae2199 Merge branch 'dd/upload-pack-stateless-eof' into seen $ make clean >/dev/null $ $ make CC=cgcc >sout2 2>&1 $ ./git version git version 2.29.2.389.g4141ae2199 $ git describe v2.29.2-389-g4141ae2199 $ $ grep 'warn' sout2 pack-revindex.c:66:23: warning: memset with byte count of 262144 upload-pack.c:1114:86: warning: Using plain integer as NULL pointer $ So, this would have failed, because some extra flags ('SP_EXTRA_FLAGS' to be precise) are not being passed to sparse. We can see that: $ grep SP_EXTRA_FLAGS Makefile SP_EXTRA_FLAGS = -Wno-universal-initializer http-push.sp http.sp http-walker.sp remote-curl.sp imap-send.sp: SP_EXTRA_FLAGS += \ pack-revindex.sp: SP_EXTRA_FLAGS += -Wno-memcpy-max-count compat/nedmalloc/nedmalloc.sp: SP_EXTRA_FLAGS += -Wno-non-pointer-null $(SPARSE_FLAGS) $(SP_EXTRA_FLAGS) $< $ (Note that you can't see -DCURL_DISABLE_TYPECHECK being set for all the libcurl using files). So, either sparse has improved to the point that the 'typecheck' shenanigans used in the libcurl headers don't cause it problems anymore, or the headers have changed sufficiently. (Hmm, that header doesn't seem to have changed much). I don't build with nedmalloc, but last time I did it was littered with uses of '0' as a NULL pointer (hence the -Wno-non-pointer-null). Given that we will probably not re-import that code, I suppose all of those sparse warnings could be fixed. Which leaves 'pack-revindex.c'. Unfortunately, I don't see an easy fix for that (-Wno-universal-initializer is now the default, so we don't need that anymore). Well, I suppose anybody could add 'CFLAGS+=-Wno-\ memcpy-max-count' to their config.mak file (this '-W' argument is filtered out by cgcc and not passed to gcc). [EDIT: I just tried this and, yes, this does work! I'm still surprised about curl headers ;-) ] Note also that my build slowed down by 14% using CC=cgcc, since it is running both sparse and gcc on each source file. I guess it would be most useful on a CI build, but I don't know what would be involved in setting that up. ATB, Ramsay Jones [1] It took about one month for me to get used to the 'pu'->'seen' change, but that was mostly training my fingers to type all of the output files that are keyed by the branch name: pout->sout, psp-out->ssp-out, psc->ssc, phcout->shcout and ptest-out->stest-out. :D
Ramsay Jones <ramsay@ramsayjones.plus.com> writes: > Yes, anybody who just does 'make sparse' will notice the failure, so > that would be a definite improvement. (How many people run 'make sparse' > though?). At least, I was planning to add one to the number when I wrote the patch, so that I would start using it as part of daily integration build before pushing day's result out. I can tweak my $(MAKE) command line arguments in the script I use for daily integration builds, so the patch is not strictly needed for me personally. > I guess it would be most useful on a CI build, but I don't know what > would be involved in setting that up. CI builds already have enough stuff around invocation of "make test" etc., and it would be trivial to pass SPARSE_FLAGS from the command line when adding "make sparse" invocation to one of the scripts in ci/ directory, so from that point of view, this patch is not needed for them, either. My hope was that it would make it harder for individual contributors who run "make sparse" and fail to notice an error from it.
On 02/11/2020 18:55, Junio C Hamano wrote: > Ramsay Jones <ramsay@ramsayjones.plus.com> writes: > >> Yes, anybody who just does 'make sparse' will notice the failure, so >> that would be a definite improvement. (How many people run 'make sparse' >> though?). > > At least, I was planning to add one to the number when I wrote the > patch, so that I would start using it as part of daily integration > build before pushing day's result out. Great! :-D How are you installing sparse? (I am using Linux Mint 20, based on Ubuntu 20.04, whose sparse package version is 0.6.1-2build1 - which is to say too old; you require version 0.6.2 or greater. Building from source, I am currently using v0.6.3-76-gf680124b). [I seem to remember you saying you used Fedora, which usually has more up-to-date software than most distros]. >> I guess it would be most useful on a CI build, but I don't know what >> would be involved in setting that up. > > CI builds already have enough stuff around invocation of "make test" > etc., and it would be trivial to pass SPARSE_FLAGS from the command > line when adding "make sparse" invocation to one of the scripts in > ci/ directory, so from that point of view, this patch is not needed > for them, either. My concern was more about how the CI system obtains/installs/builds a sufficiently new version of sparse. Otherwise, 'make sparse' won't do very much. ;-) As I said, I don't know what's involved in getting that to work. ATB, Ramsay Jones
Ramsay Jones <ramsay@ramsayjones.plus.com> writes: > How are you installing sparse? (I am using Linux Mint 20, based on > Ubuntu 20.04, whose sparse package version is 0.6.1-2build1 - which > is to say too old; you require version 0.6.2 or greater. Building > from source, I am currently using v0.6.3-76-gf680124b). Built and installed in ~/$arch/gitstuff/bin which is on my $PATH [remote "origin"] url = git://git.kernel.org/pub/scm/devel/sparse/sparse.git > My concern was more about how the CI system obtains/installs/builds a > sufficiently new version of sparse. Otherwise, 'make sparse' won't do > very much. ;-) As I said, I don't know what's involved in getting > that to work. Ah, yes, it would be a disaster to trigger false positive by using old version like 0.6.1, so we may end up having to bring our own version in, which is more work X-<. Thanks.
Hi Ramsay, On Tue, 3 Nov 2020, Ramsay Jones wrote: > On 02/11/2020 18:55, Junio C Hamano wrote: > > > CI builds already have enough stuff around invocation of "make test" > > etc., and it would be trivial to pass SPARSE_FLAGS from the command > > line when adding "make sparse" invocation to one of the scripts in ci/ > > directory, so from that point of view, this patch is not needed for > > them, either. > > My concern was more about how the CI system obtains/installs/builds a > sufficiently new version of sparse. Otherwise, 'make sparse' won't do > very much. ;-) As I said, I don't know what's involved in getting that > to work. As mentioned in https://github.com/gitgitgadget/git/issues/345, there is a Pipeline that builds sparse packages for Ubuntu, ready for use in our CI build: https://dev.azure.com/git/git/_build?definitionId=10&_a=summary. Currently, it is scheduled to run every weekday at 5:00am (I assume that's UTC). So I went ahead and added more details to the ticket, essentially demonstrating how the `sparse` package can be downloaded and installed in the CI build in a Bash step: -- snip -- urlbase=https://dev.azure.com/git/git/_apis/build/builds id=$(curl "$urlbase?definitions=10&statusFilter=completed&resultFilter=succeeded&\$top=1" | json_pp | sed -n 's/^ "id" : \([1-9][0-9]*\).*/\1/p') download_url="$(curl "$urlbase/$id/artifacts" | json_pp | sed -n '/^ {/{:1;N;/\n }/b2;b1;:2;/"name" : "sparse"/{s/.*"downloadUrl" : "\([^"]*\).*/\1/p}}')" curl -Lo sparse.zip "$download_url" unzip sparse.zip sudo dpkg -i sparse/*.deb -- snap -- It is quite likely that the ugly and unwieldy `json_pp | sed` things could be replaced by elegant `jq` calls. Ciao, Dscho
On 04/11/2020 13:43, Johannes Schindelin wrote: > Hi Ramsay, > > On Tue, 3 Nov 2020, Ramsay Jones wrote: > >> On 02/11/2020 18:55, Junio C Hamano wrote: >> >>> CI builds already have enough stuff around invocation of "make test" >>> etc., and it would be trivial to pass SPARSE_FLAGS from the command >>> line when adding "make sparse" invocation to one of the scripts in ci/ >>> directory, so from that point of view, this patch is not needed for >>> them, either. >> >> My concern was more about how the CI system obtains/installs/builds a >> sufficiently new version of sparse. Otherwise, 'make sparse' won't do >> very much. ;-) As I said, I don't know what's involved in getting that >> to work. > > As mentioned in https://github.com/gitgitgadget/git/issues/345, there is a > Pipeline that builds sparse packages for Ubuntu, ready for use in our CI > build: https://dev.azure.com/git/git/_build?definitionId=10&_a=summary. > Currently, it is scheduled to run every weekday at 5:00am (I assume that's > UTC). Oh-my! Has there been a CI 'make sparse' build since september last year? :-D I missed that! (I couldn't view the azure build summary given above - it just keeps asking for a user/password :( ). ATB, Ramsay Jones
Ramsay Jones <ramsay@ramsayjones.plus.com> writes: > On 04/11/2020 13:43, Johannes Schindelin wrote: >> Hi Ramsay, >> >> On Tue, 3 Nov 2020, Ramsay Jones wrote: >> >>> On 02/11/2020 18:55, Junio C Hamano wrote: >>> >>>> CI builds already have enough stuff around invocation of "make test" >>>> etc., and it would be trivial to pass SPARSE_FLAGS from the command >>>> line when adding "make sparse" invocation to one of the scripts in ci/ >>>> directory, so from that point of view, this patch is not needed for >>>> them, either. >>> >>> My concern was more about how the CI system obtains/installs/builds a >>> sufficiently new version of sparse. Otherwise, 'make sparse' won't do >>> very much. ;-) As I said, I don't know what's involved in getting that >>> to work. >> >> As mentioned in https://github.com/gitgitgadget/git/issues/345, there is a >> Pipeline that builds sparse packages for Ubuntu, ready for use in our CI >> build: https://dev.azure.com/git/git/_build?definitionId=10&_a=summary. >> Currently, it is scheduled to run every weekday at 5:00am (I assume that's >> UTC). > > Oh-my! Has there been a CI 'make sparse' build since september last year? :-D > I missed that! (I couldn't view the azure build summary given above - it just > keeps asking for a user/password :( ). Hmph, is that what Dscho said? I thought he was just saying packaged versions of latest sparse usable on Ubuntu are regularly available so those who wants to add "make sparse" to our jobs have a place to fetch it from.
On 04/11/2020 18:11, Junio C Hamano wrote: > Ramsay Jones <ramsay@ramsayjones.plus.com> writes: > >> On 04/11/2020 13:43, Johannes Schindelin wrote: >>> Hi Ramsay, >>> >>> On Tue, 3 Nov 2020, Ramsay Jones wrote: >>> >>>> On 02/11/2020 18:55, Junio C Hamano wrote: >>>> >>>>> CI builds already have enough stuff around invocation of "make test" >>>>> etc., and it would be trivial to pass SPARSE_FLAGS from the command >>>>> line when adding "make sparse" invocation to one of the scripts in ci/ >>>>> directory, so from that point of view, this patch is not needed for >>>>> them, either. >>>> >>>> My concern was more about how the CI system obtains/installs/builds a >>>> sufficiently new version of sparse. Otherwise, 'make sparse' won't do >>>> very much. ;-) As I said, I don't know what's involved in getting that >>>> to work. >>> >>> As mentioned in https://github.com/gitgitgadget/git/issues/345, there is a >>> Pipeline that builds sparse packages for Ubuntu, ready for use in our CI >>> build: https://dev.azure.com/git/git/_build?definitionId=10&_a=summary. >>> Currently, it is scheduled to run every weekday at 5:00am (I assume that's >>> UTC). >> >> Oh-my! Has there been a CI 'make sparse' build since september last year? :-D >> I missed that! (I couldn't view the azure build summary given above - it just >> keeps asking for a user/password :( ). > > Hmph, is that what Dscho said? > > I thought he was just saying packaged versions of latest sparse > usable on Ubuntu are regularly available so those who wants to add > "make sparse" to our jobs have a place to fetch it from. Ah, yes. Having read that again, I think your interpretation is correct. Ahem. My mistake. Still, it's good to know that a suitable version of sparse is available should anyone need it. Thanks! ATB, Ramsay Jones
diff --git c/config.mak.dev w/config.mak.dev index 3126a5364d..022fb58218 100644 --- c/config.mak.dev +++ w/config.mak.dev @@ -1,5 +1,6 @@ ifeq ($(filter no-error,$(DEVOPTS)),) DEVELOPER_CFLAGS += -Werror +SPARSE_FLAGS += -Wsparse-error endif ifneq ($(filter pedantic,$(DEVOPTS)),) DEVELOPER_CFLAGS += -pedantic
With -Wsparse-error, "make sparse" would fail, instead of just giving a warning message. Signed-off-by: Junio C Hamano <gitster@pobox.com> --- config.mak.dev | 1 + 1 file changed, 1 insertion(+)