| Message ID | 525669b3b38ed57d6d4f188dfe0bb8fe10b63749.1569227313.git.gitgitgadget@gmail.com (mailing list archive) |
|---|---|
| State | New, archived |
| Headers | show |
| Series | contrib/buildsystems: fix Visual Studio Debug configuration | expand |
Johannes, would you please review?
Hi Alexandr, On Mon, 23 Sep 2019, Alexandr Miloslavskiy via GitGitGadget wrote: > From: Alexandr Miloslavskiy <alexandr.miloslavskiy@syntevo.com> > > Even though Debug configuration builds, the resulting build is incorrect > in a subtle way: it mixes up Debug and Release binaries, which in turn > causes hard-to-predict bugs. > > In my case, when git calls iconv library, iconv sets 'errno' and git > then tests it, but in Debug and Release CRT those 'errno' are different > memory locations. > > This patch addresses 3 connected bugs: > 1) Typo in '\(Configuration)'. As a result, Debug configuration > condition is always false and Release path is taken instead. > 2) Regexp that replaced 'zlib.lib' with 'zlibd.lib' was only affecting > the first occurrence. However, some projects have it listed twice. > Previously this bug was hidden, because Debug path was never taken. > I decided that avoiding double -lz in makefile is fragile and I'd > better replace all occurrences instead. > 3) In Debug, 'libcurl-d.lib' should be used instead of 'libcurl.lib'. > Previously this bug was hidden, because Debug path was never taken. > > Signed-off-by: Alexandr Miloslavskiy <alexandr.miloslavskiy@syntevo.com> ACK! Johannes > --- > contrib/buildsystems/Generators/Vcxproj.pm | 15 ++++++++------- > 1 file changed, 8 insertions(+), 7 deletions(-) > > diff --git a/contrib/buildsystems/Generators/Vcxproj.pm b/contrib/buildsystems/Generators/Vcxproj.pm > index 576ccabe1d..7b1e277eca 100644 > --- a/contrib/buildsystems/Generators/Vcxproj.pm > +++ b/contrib/buildsystems/Generators/Vcxproj.pm > @@ -79,7 +79,8 @@ sub createProject { > if (!$static_library) { > $libs_release = join(";", sort(grep /^(?!libgit\.lib|xdiff\/lib\.lib|vcs-svn\/lib\.lib)/, @{$$build_structure{"$prefix${name}_LIBS"}})); > $libs_debug = $libs_release; > - $libs_debug =~ s/zlib\.lib/zlibd\.lib/; > + $libs_debug =~ s/zlib\.lib/zlibd\.lib/g; > + $libs_debug =~ s/libcurl\.lib/libcurl-d\.lib/g; > } > > $defines =~ s/-D//g; > @@ -119,13 +120,13 @@ sub createProject { > <VCPKGArch Condition="'\$(Platform)'=='Win32'">x86-windows</VCPKGArch> > <VCPKGArch Condition="'\$(Platform)'!='Win32'">x64-windows</VCPKGArch> > <VCPKGArchDirectory>$cdup\\compat\\vcbuild\\vcpkg\\installed\\\$(VCPKGArch)</VCPKGArchDirectory> > - <VCPKGBinDirectory Condition="'\(Configuration)'=='Debug'">\$(VCPKGArchDirectory)\\debug\\bin</VCPKGBinDirectory> > - <VCPKGLibDirectory Condition="'\(Configuration)'=='Debug'">\$(VCPKGArchDirectory)\\debug\\lib</VCPKGLibDirectory> > - <VCPKGBinDirectory Condition="'\(Configuration)'!='Debug'">\$(VCPKGArchDirectory)\\bin</VCPKGBinDirectory> > - <VCPKGLibDirectory Condition="'\(Configuration)'!='Debug'">\$(VCPKGArchDirectory)\\lib</VCPKGLibDirectory> > + <VCPKGBinDirectory Condition="'\$(Configuration)'=='Debug'">\$(VCPKGArchDirectory)\\debug\\bin</VCPKGBinDirectory> > + <VCPKGLibDirectory Condition="'\$(Configuration)'=='Debug'">\$(VCPKGArchDirectory)\\debug\\lib</VCPKGLibDirectory> > + <VCPKGBinDirectory Condition="'\$(Configuration)'!='Debug'">\$(VCPKGArchDirectory)\\bin</VCPKGBinDirectory> > + <VCPKGLibDirectory Condition="'\$(Configuration)'!='Debug'">\$(VCPKGArchDirectory)\\lib</VCPKGLibDirectory> > <VCPKGIncludeDirectory>\$(VCPKGArchDirectory)\\include</VCPKGIncludeDirectory> > - <VCPKGLibs Condition="'\(Configuration)'=='Debug'">$libs_debug</VCPKGLibs> > - <VCPKGLibs Condition="'\(Configuration)'!='Debug'">$libs_release</VCPKGLibs> > + <VCPKGLibs Condition="'\$(Configuration)'=='Debug'">$libs_debug</VCPKGLibs> > + <VCPKGLibs Condition="'\$(Configuration)'!='Debug'">$libs_release</VCPKGLibs> > </PropertyGroup> > <Import Project="\$(VCTargetsPath)\\Microsoft.Cpp.Default.props" /> > <PropertyGroup Condition="'\$(Configuration)'=='Debug'" Label="Configuration"> > -- > gitgitgadget >
On 23/09/2019 09:28, Alexandr Miloslavskiy via GitGitGadget wrote: > From: Alexandr Miloslavskiy <alexandr.miloslavskiy@syntevo.com> > > Even though Debug configuration builds, the resulting build is incorrect > in a subtle way: it mixes up Debug and Release binaries, which in turn > causes hard-to-predict bugs. > > In my case, when git calls iconv library, iconv sets 'errno' and git > then tests it, but in Debug and Release CRT those 'errno' are different > memory locations. > > This patch addresses 3 connected bugs: > 1) Typo in '\(Configuration)'. As a result, Debug configuration > condition is always false and Release path is taken instead. > 2) Regexp that replaced 'zlib.lib' with 'zlibd.lib' was only affecting > the first occurrence. However, some projects have it listed twice. > Previously this bug was hidden, because Debug path was never taken. > I decided that avoiding double -lz in makefile is fragile and I'd > better replace all occurrences instead. > 3) In Debug, 'libcurl-d.lib' should be used instead of 'libcurl.lib'. > Previously this bug was hidden, because Debug path was never taken. I just bumped against a potential issue like this. I was test compiling [1a,b] the `vs/master` branch from Git-For-Windows and got the LINK error that the 'libcurl-d.lib' was not found (4 places). Error LNK1104 cannot open file 'libcurl-d.lib' git-imap-send C:\git-sdk-64\usr\src\git\git-imap-send\ LINK 1 Having just located this email, I changed the build type to 'Release' and the errors disappeared. Do we also need to identify where the libcurl-d.lib will be found? i.e. is it something that needs including via the sdk pacman list (I think I'm up to date but maybe not..) A quick web search didn't show any hits for `libcurl-d.lib` (with the dash `-`), though did find a few for `libcurld.lib`. Philip Why compiling: [1a] https://www.sourcetrail.com is a cross-platform source explorer that helps you get productive on unfamiliar source code. [1b] Sourcetrail is now free and open-source https://www.sourcetrail.com/blog/open_source/ > > Signed-off-by: Alexandr Miloslavskiy <alexandr.miloslavskiy@syntevo.com> > --- > contrib/buildsystems/Generators/Vcxproj.pm | 15 ++++++++------- > 1 file changed, 8 insertions(+), 7 deletions(-) > > diff --git a/contrib/buildsystems/Generators/Vcxproj.pm b/contrib/buildsystems/Generators/Vcxproj.pm > index 576ccabe1d..7b1e277eca 100644 > --- a/contrib/buildsystems/Generators/Vcxproj.pm > +++ b/contrib/buildsystems/Generators/Vcxproj.pm > @@ -79,7 +79,8 @@ sub createProject { > if (!$static_library) { > $libs_release = join(";", sort(grep /^(?!libgit\.lib|xdiff\/lib\.lib|vcs-svn\/lib\.lib)/, @{$$build_structure{"$prefix${name}_LIBS"}})); > $libs_debug = $libs_release; > - $libs_debug =~ s/zlib\.lib/zlibd\.lib/; > + $libs_debug =~ s/zlib\.lib/zlibd\.lib/g; > + $libs_debug =~ s/libcurl\.lib/libcurl-d\.lib/g; > } > > $defines =~ s/-D//g; > @@ -119,13 +120,13 @@ sub createProject { > <VCPKGArch Condition="'\$(Platform)'=='Win32'">x86-windows</VCPKGArch> > <VCPKGArch Condition="'\$(Platform)'!='Win32'">x64-windows</VCPKGArch> > <VCPKGArchDirectory>$cdup\\compat\\vcbuild\\vcpkg\\installed\\\$(VCPKGArch)</VCPKGArchDirectory> > - <VCPKGBinDirectory Condition="'\(Configuration)'=='Debug'">\$(VCPKGArchDirectory)\\debug\\bin</VCPKGBinDirectory> > - <VCPKGLibDirectory Condition="'\(Configuration)'=='Debug'">\$(VCPKGArchDirectory)\\debug\\lib</VCPKGLibDirectory> > - <VCPKGBinDirectory Condition="'\(Configuration)'!='Debug'">\$(VCPKGArchDirectory)\\bin</VCPKGBinDirectory> > - <VCPKGLibDirectory Condition="'\(Configuration)'!='Debug'">\$(VCPKGArchDirectory)\\lib</VCPKGLibDirectory> > + <VCPKGBinDirectory Condition="'\$(Configuration)'=='Debug'">\$(VCPKGArchDirectory)\\debug\\bin</VCPKGBinDirectory> > + <VCPKGLibDirectory Condition="'\$(Configuration)'=='Debug'">\$(VCPKGArchDirectory)\\debug\\lib</VCPKGLibDirectory> > + <VCPKGBinDirectory Condition="'\$(Configuration)'!='Debug'">\$(VCPKGArchDirectory)\\bin</VCPKGBinDirectory> > + <VCPKGLibDirectory Condition="'\$(Configuration)'!='Debug'">\$(VCPKGArchDirectory)\\lib</VCPKGLibDirectory> > <VCPKGIncludeDirectory>\$(VCPKGArchDirectory)\\include</VCPKGIncludeDirectory> > - <VCPKGLibs Condition="'\(Configuration)'=='Debug'">$libs_debug</VCPKGLibs> > - <VCPKGLibs Condition="'\(Configuration)'!='Debug'">$libs_release</VCPKGLibs> > + <VCPKGLibs Condition="'\$(Configuration)'=='Debug'">$libs_debug</VCPKGLibs> > + <VCPKGLibs Condition="'\$(Configuration)'!='Debug'">$libs_release</VCPKGLibs> > </PropertyGroup> > <Import Project="\$(VCTargetsPath)\\Microsoft.Cpp.Default.props" /> > <PropertyGroup Condition="'\$(Configuration)'=='Debug'" Label="Configuration">
On 27.11.2019 18:56, Philip Oakley wrote: > I just bumped against a potential issue like this. I was test compiling > [1a,b] the `vs/master` branch from Git-For-Windows and got the LINK > error that the 'libcurl-d.lib' was not found (4 places). > > Error LNK1104 cannot open file 'libcurl-d.lib' git-imap-send > C:\git-sdk-64\usr\src\git\git-imap-send\ LINK 1 > > Having just located this email, I changed the build type to 'Release' > and the errors disappeared. > > Do we also need to identify where the libcurl-d.lib will be found? i.e. > is it something that needs including via the sdk pacman list (I think > I'm up to date but maybe not..) > > A quick web search didn't show any hits for `libcurl-d.lib` (with the > dash `-`), though did find a few for `libcurld.lib`. If you clone `git-for-windows` and build in VS using `git.sln`, it will automatically clone `git-for-windows\compat\vcbuild\vcpkg` and build various things there, resulting in `git-for-windows\compat\vcbuild\vcpkg\buildtrees\curl\x64-windows-dbg\lib\libcurl-d.dll` `git-for-windows\compat\vcbuild\vcpkg\installed\x64-windows\debug\bin\libcurl-d.dll` `git-for-windows\compat\vcbuild\vcpkg\packages\curl_x64-windows\debug\bin\libcurl-d.dll` Which will be picked up by solution to build git. I have built Debug many times now and didn't have any issues. If you do, I would suggest to clone a new copy and build it.
Hi Alexandr, On 27/11/2019 18:59, Alexandr Miloslavskiy wrote: > On 27.11.2019 18:56, Philip Oakley wrote: >> I just bumped against a potential issue like this. I was test >> compiling [1a,b] the `vs/master` branch from Git-For-Windows and got >> the LINK error that the 'libcurl-d.lib' was not found (4 places). >> >> Error LNK1104 cannot open file 'libcurl-d.lib' git-imap-send >> C:\git-sdk-64\usr\src\git\git-imap-send\ LINK 1 >> >> Having just located this email, I changed the build type to 'Release' >> and the errors disappeared. >> >> Do we also need to identify where the libcurl-d.lib will be found? >> i.e. is it something that needs including via the sdk pacman list (I >> think I'm up to date but maybe not..) >> >> A quick web search didn't show any hits for `libcurl-d.lib` (with the >> dash `-`), though did find a few for `libcurld.lib`. > > If you clone `git-for-windows` and build in VS using `git.sln`, it > will automatically clone `git-for-windows\compat\vcbuild\vcpkg` and > build various things there, resulting in > > `git-for-windows\compat\vcbuild\vcpkg\buildtrees\curl\x64-windows-dbg\lib\libcurl-d.dll` > > > `git-for-windows\compat\vcbuild\vcpkg\installed\x64-windows\debug\bin\libcurl-d.dll` > > > `git-for-windows\compat\vcbuild\vcpkg\packages\curl_x64-windows\debug\bin\libcurl-d.dll` > > > Which will be picked up by solution to build git. > > I have built Debug many times now and didn't have any issues. If you > do, I would suggest to clone a new copy and build it. I already have the clone of vcpkg from 30/03/2019 with those three files already present. Have they been updated since then? The error report is specifically that the .lib file is missing (which I can't find..) However the plain `libcurl.lib` is present with matching dll's. It just feels a little unusual. On a side note there's a few (not many) Stackoverflow questions about building libcurl-d.lib, but they feel almost tangential https://stackoverflow.com/questions/51418238/libcurl-a-debug-lib-libcurl-a-lib-are-generated-instead-of-libcurld-lib-libcur https://stackoverflow.com/questions/37126943/libcurld-dll-is-missing-from-your-computer-adding-libcurl-to-visual-studio-pr quick look.. not sure if the assertion about choosing "ONE" (of debug/release) in https://github.com/git-for-windows/git/blob/master/compat/vcbuild/README#L26-L30 is part of the issue. That README doesn't really cover the update method for the vcpkg repository. It presumes you start from a fresh clone, which can be slow. I fetched/pulled the vcpkg repoo but no sign of an update. Philip
On 27/11/2019 18:59, Alexandr Miloslavskiy wrote: > On 27.11.2019 18:56, Philip Oakley wrote: >> I just bumped against a potential issue like this. I was test >> compiling [1a,b] the `vs/master` branch from Git-For-Windows and got >> the LINK error that the 'libcurl-d.lib' was not found (4 places). >> >> Error LNK1104 cannot open file 'libcurl-d.lib' git-imap-send >> C:\git-sdk-64\usr\src\git\git-imap-send\ LINK 1 >> >> Having just located this email, I changed the build type to 'Release' >> and the errors disappeared. >> >> Do we also need to identify where the libcurl-d.lib will be found? >> i.e. is it something that needs including via the sdk pacman list (I >> think I'm up to date but maybe not..) >> >> A quick web search didn't show any hits for `libcurl-d.lib` (with the >> dash `-`), though did find a few for `libcurld.lib`. > > If you clone `git-for-windows` and build in VS using `git.sln`, it > will automatically clone `git-for-windows\compat\vcbuild\vcpkg` and > build various things there, resulting in > > `git-for-windows\compat\vcbuild\vcpkg\buildtrees\curl\x64-windows-dbg\lib\libcurl-d.dll` > > > `git-for-windows\compat\vcbuild\vcpkg\installed\x64-windows\debug\bin\libcurl-d.dll` > > > `git-for-windows\compat\vcbuild\vcpkg\packages\curl_x64-windows\debug\bin\libcurl-d.dll` > > > Which will be picked up by solution to build git. > > I have built Debug many times now and didn't have any issues. If you > do, I would suggest to clone a new copy and build it. Hmm, 45 minutes of cloning and rebuild, but finally it compiled clean (both Release and Debug)
On 28.11.2019 1:34, Philip Oakley wrote: > Hmm, 45 minutes of cloning and rebuild, but finally it compiled clean > (both Release and Debug) I understand that the issue is resolved now. Probably your old repo was missing the libraries for whatever reason (like antivirus deleting them, etc), but build script thought that dependencies are properly built, so didn't attempt to rebuild them.
On 28/11/2019 10:07, Alexandr Miloslavskiy wrote: > On 28.11.2019 1:34, Philip Oakley wrote: > >> Hmm, 45 minutes of cloning and rebuild, but finally it compiled clean >> (both Release and Debug) > > I understand that the issue is resolved now. > > Probably your old repo was missing the libraries for whatever reason > (like antivirus deleting them, etc), but build script thought that > dependencies are properly built, so didn't attempt to rebuild them. I think I'll report that as an issue to the https://github.com/microsoft/vcpkg folks so that there's better detection for 'out of date' / updated vcpkg issues. The vcpkg_install.bat in /compat/vcbuild may need updating to do a 'pull' if there is an existing directory. At the moment it's a rather simplistic 'all or nothing' for getting all those extra packages. Philip
On 29/11/2019 11:53, Philip Oakley wrote: > On 28/11/2019 10:07, Alexandr Miloslavskiy wrote: >> On 28.11.2019 1:34, Philip Oakley wrote: >> >>> Hmm, 45 minutes of cloning and rebuild, but finally it compiled >>> clean (both Release and Debug) >> >> I understand that the issue is resolved now. >> >> Probably your old repo was missing the libraries for whatever reason >> (like antivirus deleting them, etc), but build script thought that >> dependencies are properly built, so didn't attempt to rebuild them. > I think I'll report that as an issue to the > https://github.com/microsoft/vcpkg folks so that there's better > detection for 'out of date' / updated vcpkg issues. > > The vcpkg_install.bat in /compat/vcbuild may need updating to do a > 'pull' if there is an existing directory. At the moment it's a rather > simplistic 'all or nothing' for getting all those extra packages. Issue submitted as https://github.com/microsoft/vcpkg/issues/9148 -- Philip
On 28/11/2019 10:07, Alexandr Miloslavskiy wrote: > On 28.11.2019 1:34, Philip Oakley wrote: > >> Hmm, 45 minutes of cloning and rebuild, but finally it compiled clean >> (both Release and Debug) > > I understand that the issue is resolved now. > > Probably your old repo was missing the libraries for whatever reason > (like antivirus deleting them, etc), but build script thought that > dependencies are properly built, so didn't attempt to rebuild them. I was searching for how Visual Studio managed to decide if the vcpkg needed installing. I think I've found it in 384a61bc6a ("contrib/buildsystems: add a backend for modern Visual Studio versions", 2019-07-29) where dscho says: we initialize the `vcpkg` conditionally, in the `libgit` project's `PreBuildEvent`. To allow for parallel building of the projects, we therefore put `libgit` at the bottom of the project hierarchy. What's not clear is if the conditional pre-build can have an if-else option so that we can have an 'update' check if already installed. Adding the vcpkg update to the vcpkg_install.bat didn't work :-( Philip PS. the https://github.com/CoatiSoftware/Sourcetrail visualiser is looking nice.
diff --git a/contrib/buildsystems/Generators/Vcxproj.pm b/contrib/buildsystems/Generators/Vcxproj.pm index 576ccabe1d..7b1e277eca 100644 --- a/contrib/buildsystems/Generators/Vcxproj.pm +++ b/contrib/buildsystems/Generators/Vcxproj.pm @@ -79,7 +79,8 @@ sub createProject { if (!$static_library) { $libs_release = join(";", sort(grep /^(?!libgit\.lib|xdiff\/lib\.lib|vcs-svn\/lib\.lib)/, @{$$build_structure{"$prefix${name}_LIBS"}})); $libs_debug = $libs_release; - $libs_debug =~ s/zlib\.lib/zlibd\.lib/; + $libs_debug =~ s/zlib\.lib/zlibd\.lib/g; + $libs_debug =~ s/libcurl\.lib/libcurl-d\.lib/g; } $defines =~ s/-D//g; @@ -119,13 +120,13 @@ sub createProject { <VCPKGArch Condition="'\$(Platform)'=='Win32'">x86-windows</VCPKGArch> <VCPKGArch Condition="'\$(Platform)'!='Win32'">x64-windows</VCPKGArch> <VCPKGArchDirectory>$cdup\\compat\\vcbuild\\vcpkg\\installed\\\$(VCPKGArch)</VCPKGArchDirectory> - <VCPKGBinDirectory Condition="'\(Configuration)'=='Debug'">\$(VCPKGArchDirectory)\\debug\\bin</VCPKGBinDirectory> - <VCPKGLibDirectory Condition="'\(Configuration)'=='Debug'">\$(VCPKGArchDirectory)\\debug\\lib</VCPKGLibDirectory> - <VCPKGBinDirectory Condition="'\(Configuration)'!='Debug'">\$(VCPKGArchDirectory)\\bin</VCPKGBinDirectory> - <VCPKGLibDirectory Condition="'\(Configuration)'!='Debug'">\$(VCPKGArchDirectory)\\lib</VCPKGLibDirectory> + <VCPKGBinDirectory Condition="'\$(Configuration)'=='Debug'">\$(VCPKGArchDirectory)\\debug\\bin</VCPKGBinDirectory> + <VCPKGLibDirectory Condition="'\$(Configuration)'=='Debug'">\$(VCPKGArchDirectory)\\debug\\lib</VCPKGLibDirectory> + <VCPKGBinDirectory Condition="'\$(Configuration)'!='Debug'">\$(VCPKGArchDirectory)\\bin</VCPKGBinDirectory> + <VCPKGLibDirectory Condition="'\$(Configuration)'!='Debug'">\$(VCPKGArchDirectory)\\lib</VCPKGLibDirectory> <VCPKGIncludeDirectory>\$(VCPKGArchDirectory)\\include</VCPKGIncludeDirectory> - <VCPKGLibs Condition="'\(Configuration)'=='Debug'">$libs_debug</VCPKGLibs> - <VCPKGLibs Condition="'\(Configuration)'!='Debug'">$libs_release</VCPKGLibs> + <VCPKGLibs Condition="'\$(Configuration)'=='Debug'">$libs_debug</VCPKGLibs> + <VCPKGLibs Condition="'\$(Configuration)'!='Debug'">$libs_release</VCPKGLibs> </PropertyGroup> <Import Project="\$(VCTargetsPath)\\Microsoft.Cpp.Default.props" /> <PropertyGroup Condition="'\$(Configuration)'=='Debug'" Label="Configuration">