| Message ID | 20190718215428.6727-1-george.dunlap@citrix.com (mailing list archive) |
|---|---|
| State | New, archived |
| Headers | show |
| Series | golang/xenlight: Add libxl_utils support | expand |
On 7/18/19 11:54 PM, George Dunlap wrote: > The Go bindings for libxl miss functions from libxl_utils, let's start > with the simple libxl_domid_to_name and its counterpart > libxl_name_to_domid. > > NB that C.GoString() will return "" if it's passed a NULL; see > https://github.com/golang/go/issues/32734#issuecomment-506835432 > > Signed-off-by: Nicolas Belouin <nicolas.belouin@gandi.net> > Signed-off-by: George Dunlap <george.dunlap@citrix.com> > --- > v3: > - Wire into build system > - Add reference to C.GoString() handling NULL to commit message > > Nicolas, could you test to see if this actually works for you? Tested it, it works. I must confess I do not use that import path as the new modules mechanism introduced in Go1.11 downloads and compile a versioned copy of every dependency per project, and this behavior is incompatible with the build system used here. Nicolas
> On Jul 19, 2019, at 8:34 AM, Nicolas Belouin <nicolas.belouin@gandi.net> wrote: > > > > On 7/18/19 11:54 PM, George Dunlap wrote: >> The Go bindings for libxl miss functions from libxl_utils, let's start >> with the simple libxl_domid_to_name and its counterpart >> libxl_name_to_domid. >> >> NB that C.GoString() will return "" if it's passed a NULL; see >> https://github.com/golang/go/issues/32734#issuecomment-506835432 >> >> Signed-off-by: Nicolas Belouin <nicolas.belouin@gandi.net> >> Signed-off-by: George Dunlap <george.dunlap@citrix.com> >> --- >> v3: >> - Wire into build system >> - Add reference to C.GoString() handling NULL to commit message >> >> Nicolas, could you test to see if this actually works for you? > Tested it, it works. > > I must confess I do not use that import path as the new modules mechanism > introduced in Go1.11 downloads and compile a versioned copy of every > dependency per project, and this behavior is incompatible with the build > system used here. It’s possible that something fundamentally has changed, but I suspect that rather you don’t quite understand how the current build system is supposed to work. (In which case a write-up in the tree would probably be useful.) Go has always insisted that there be no binary compatibility between versions; so it’s always been necessary to re-compile all your libraries when upgrading from (say) 1.8 to 1.9. Which means that any useable distribution must also include all the source files necessary to recompile when you bump the version number. So the core mechanism of the “install” is actually to copy all the source files necessary into the right local directory such that the go compiler can find them; ATM this is /usr/share/gocode/golang.xenproject.org/xenlight (as documented here: https://github.com/golang/go/wiki/FileTreeDocumentation). The files which are copied are specified by the PGKSOURCES variable in the makefile. The purpose of building a binary during the normal “make” process is not actually to have a useable binary; it’s to detect any potential issues when you do the Xen build, rather than whenever you actually build a Go program for the first time (potentially on some end-user’s system). And “detect any issues” attempts to use “go build” in the situation you expect to use it at run time (i.e., it copies PKGSOURCES to a local directory, then points GO_PATH at them, points to the locally-built libraries, and then runs a plain `go build`). In this way it should, theoretically, catch any of the following errors: * Syntax / whatever errors in the Go bindings themselves * Mismatches between the Go bindings and libxl &c * Missing system libraries * Something not right in the installation path system All that to say — the “real” way to enable a new file in the xenlight package is to add it to PKGSOURCES. -George
> On Jul 19, 2019, at 9:47 AM, George Dunlap <george.dunlap@citrix.com> wrote: > > > >> On Jul 19, 2019, at 8:34 AM, Nicolas Belouin <nicolas.belouin@gandi.net> wrote: >> >> >> >> On 7/18/19 11:54 PM, George Dunlap wrote: >>> The Go bindings for libxl miss functions from libxl_utils, let's start >>> with the simple libxl_domid_to_name and its counterpart >>> libxl_name_to_domid. >>> >>> NB that C.GoString() will return "" if it's passed a NULL; see >>> https://github.com/golang/go/issues/32734#issuecomment-506835432 >>> >>> Signed-off-by: Nicolas Belouin <nicolas.belouin@gandi.net> >>> Signed-off-by: George Dunlap <george.dunlap@citrix.com> >>> --- >>> v3: >>> - Wire into build system >>> - Add reference to C.GoString() handling NULL to commit message >>> >>> Nicolas, could you test to see if this actually works for you? >> Tested it, it works. >> >> I must confess I do not use that import path as the new modules mechanism >> introduced in Go1.11 downloads and compile a versioned copy of every >> dependency per project, and this behavior is incompatible with the build >> system used here. > > It’s possible that something fundamentally has changed, but I suspect that rather you don’t quite understand how the current build system is supposed to work. (In which case a write-up in the tree would probably be useful.) > > Go has always insisted that there be no binary compatibility between versions; so it’s always been necessary to re-compile all your libraries when upgrading from (say) 1.8 to 1.9. Which means that any useable distribution must also include all the source files necessary to recompile when you bump the version number. > > So the core mechanism of the “install” is actually to copy all the source files necessary into the right local directory such that the go compiler can find them; ATM this is /usr/share/gocode/golang.xenproject.org/xenlight Nit: This of course should have a `src/` between `gocode/` and `golang.xenproject.org/`. NB also that this naming scheme was designed so that at some point in the future, we could actually host the xenlight packages at the URL provided. -George
On 7/19/19 8:34 AM, Nicolas Belouin wrote: > > > On 7/18/19 11:54 PM, George Dunlap wrote: >> The Go bindings for libxl miss functions from libxl_utils, let's start >> with the simple libxl_domid_to_name and its counterpart >> libxl_name_to_domid. >> >> NB that C.GoString() will return "" if it's passed a NULL; see >> https://github.com/golang/go/issues/32734#issuecomment-506835432 >> >> Signed-off-by: Nicolas Belouin <nicolas.belouin@gandi.net> >> Signed-off-by: George Dunlap <george.dunlap@citrix.com> >> --- >> v3: >> - Wire into build system >> - Add reference to C.GoString() handling NULL to commit message >> >> Nicolas, could you test to see if this actually works for you? > Tested it, it works. > > I must confess I do not use that import path as the new modules mechanism > introduced in Go1.11 downloads and compile a versioned copy of every > dependency per project, and this behavior is incompatible with the build > system used here. Also -- I can't check this in without somebody acking my changes; and you're the most obvious person to do so. :-) -George
On 7/19/19 10:50 AM, George Dunlap wrote: > >> On Jul 19, 2019, at 9:47 AM, George Dunlap <george.dunlap@citrix.com> wrote: >> >> >> >>> On Jul 19, 2019, at 8:34 AM, Nicolas Belouin <nicolas.belouin@gandi.net> wrote: >>> >>> >>> >>> On 7/18/19 11:54 PM, George Dunlap wrote: >>>> The Go bindings for libxl miss functions from libxl_utils, let's start >>>> with the simple libxl_domid_to_name and its counterpart >>>> libxl_name_to_domid. >>>> >>>> NB that C.GoString() will return "" if it's passed a NULL; see >>>> https://github.com/golang/go/issues/32734#issuecomment-506835432 >>>> >>>> Signed-off-by: Nicolas Belouin <nicolas.belouin@gandi.net> >>>> Signed-off-by: George Dunlap <george.dunlap@citrix.com> >>>> --- >>>> v3: >>>> - Wire into build system >>>> - Add reference to C.GoString() handling NULL to commit message >>>> >>>> Nicolas, could you test to see if this actually works for you? >>> Tested it, it works. >>> >>> I must confess I do not use that import path as the new modules mechanism >>> introduced in Go1.11 downloads and compile a versioned copy of every >>> dependency per project, and this behavior is incompatible with the build >>> system used here. >> It’s possible that something fundamentally has changed, but I suspect that rather you don’t quite understand how the current build system is supposed to work. (In which case a write-up in the tree would probably be useful.) >> >> Go has always insisted that there be no binary compatibility between versions; so it’s always been necessary to re-compile all your libraries when upgrading from (say) 1.8 to 1.9. Which means that any useable distribution must also include all the source files necessary to recompile when you bump the version number. >> >> So the core mechanism of the “install” is actually to copy all the source files necessary into the right local directory such that the go compiler can find them; ATM this is /usr/share/gocode/golang.xenproject.org/xenlight > Nit: This of course should have a `src/` between `gocode/` and `golang.xenproject.org/`. > > NB also that this naming scheme was designed so that at some point in the future, we could actually host the xenlight packages at the URL provided. > > -George > This new mechanism of modules is described here: https://golang.org/cmd/go/#hdr-Modules__module_versions__and_more The module system is intended to supersede the GOPATH approach and provide a way to get versioned dependencies, as such it does not rely on GOPATH at all and doesn't use sources or compiled packages present in GOPATH elements such as /usr/share/gocode and systematically fetch (at the asked version) and compile a copy of the dependency as it might be a different version from the one in GOPATH. As far as I tried, I have been unable to build my module even with the library installed. I have to use xenbits.xen.org/git-http/xen.git/tools/golang/xenlight (or one of its mirror) in order to build the module using the new mechanism (the golang.xenproject.org/xenlight works when building with modules mode disabled). Nicolas
On 7/19/19 11:03 AM, Nicolas Belouin wrote: > > > On 7/19/19 10:50 AM, George Dunlap wrote: >> >>> On Jul 19, 2019, at 9:47 AM, George Dunlap <george.dunlap@citrix.com> wrote: >>> >>> >>> >>>> On Jul 19, 2019, at 8:34 AM, Nicolas Belouin <nicolas.belouin@gandi.net> wrote: >>>> >>>> >>>> >>>> On 7/18/19 11:54 PM, George Dunlap wrote: >>>>> The Go bindings for libxl miss functions from libxl_utils, let's start >>>>> with the simple libxl_domid_to_name and its counterpart >>>>> libxl_name_to_domid. >>>>> >>>>> NB that C.GoString() will return "" if it's passed a NULL; see >>>>> https://github.com/golang/go/issues/32734#issuecomment-506835432 >>>>> >>>>> Signed-off-by: Nicolas Belouin <nicolas.belouin@gandi.net> >>>>> Signed-off-by: George Dunlap <george.dunlap@citrix.com> >>>>> --- >>>>> v3: >>>>> - Wire into build system >>>>> - Add reference to C.GoString() handling NULL to commit message >>>>> >>>>> Nicolas, could you test to see if this actually works for you? >>>> Tested it, it works. >>>> >>>> I must confess I do not use that import path as the new modules mechanism >>>> introduced in Go1.11 downloads and compile a versioned copy of every >>>> dependency per project, and this behavior is incompatible with the build >>>> system used here. >>> It’s possible that something fundamentally has changed, but I suspect that rather you don’t quite understand how the current build system is supposed to work. (In which case a write-up in the tree would probably be useful.) >>> >>> Go has always insisted that there be no binary compatibility between versions; so it’s always been necessary to re-compile all your libraries when upgrading from (say) 1.8 to 1.9. Which means that any useable distribution must also include all the source files necessary to recompile when you bump the version number. >>> >>> So the core mechanism of the “install” is actually to copy all the source files necessary into the right local directory such that the go compiler can find them; ATM this is /usr/share/gocode/golang.xenproject.org/xenlight >> Nit: This of course should have a `src/` between `gocode/` and `golang.xenproject.org/`. >> >> NB also that this naming scheme was designed so that at some point in the future, we could actually host the xenlight packages at the URL provided. >> >> -George >> > > This new mechanism of modules is described here: > https://golang.org/cmd/go/#hdr-Modules__module_versions__and_more > > The module system is intended to supersede the GOPATH approach and > provide a way to get versioned dependencies, as such > it does not rely on GOPATH at all and doesn't use sources or compiled > packages present in GOPATH elements such as /usr/share/gocode > and systematically fetch (at the asked version) and compile a copy of > the dependency as it might be a different version from the one > in GOPATH. > > As far as I tried, I have been unable to build my module even with the > library installed. > I have to use xenbits.xen.org/git-http/xen.git/tools/golang/xenlight (or > one of its mirror) in order to build the module using the new > mechanism (the golang.xenproject.org/xenlight works when building with > modules mode disabled). I took a look at the module stuff when it came out, and I was never able to make sense of how it was supposed to work. <rant>On the whole, it seems they basically hate the idea of distro packages, and seem intent on breaking them whenever people manage to start to get them working.</rant> -George
On 7/19/19 12:09 PM, George Dunlap wrote: > On 7/19/19 11:03 AM, Nicolas Belouin wrote: >> >> On 7/19/19 10:50 AM, George Dunlap wrote: >>>> On Jul 19, 2019, at 9:47 AM, George Dunlap <george.dunlap@citrix.com> wrote: >>>> >>>> >>>> >>>>> On Jul 19, 2019, at 8:34 AM, Nicolas Belouin <nicolas.belouin@gandi.net> wrote: >>>>> >>>>> >>>>> >>>>> On 7/18/19 11:54 PM, George Dunlap wrote: >>>>>> The Go bindings for libxl miss functions from libxl_utils, let's start >>>>>> with the simple libxl_domid_to_name and its counterpart >>>>>> libxl_name_to_domid. >>>>>> >>>>>> NB that C.GoString() will return "" if it's passed a NULL; see >>>>>> https://github.com/golang/go/issues/32734#issuecomment-506835432 >>>>>> >>>>>> Signed-off-by: Nicolas Belouin <nicolas.belouin@gandi.net> >>>>>> Signed-off-by: George Dunlap <george.dunlap@citrix.com> >>>>>> --- >>>>>> v3: >>>>>> - Wire into build system >>>>>> - Add reference to C.GoString() handling NULL to commit message >>>>>> >>>>>> Nicolas, could you test to see if this actually works for you? >>>>> Tested it, it works. >>>>> >>>>> I must confess I do not use that import path as the new modules mechanism >>>>> introduced in Go1.11 downloads and compile a versioned copy of every >>>>> dependency per project, and this behavior is incompatible with the build >>>>> system used here. >>>> It’s possible that something fundamentally has changed, but I suspect that rather you don’t quite understand how the current build system is supposed to work. (In which case a write-up in the tree would probably be useful.) >>>> >>>> Go has always insisted that there be no binary compatibility between versions; so it’s always been necessary to re-compile all your libraries when upgrading from (say) 1.8 to 1.9. Which means that any useable distribution must also include all the source files necessary to recompile when you bump the version number. >>>> >>>> So the core mechanism of the “install” is actually to copy all the source files necessary into the right local directory such that the go compiler can find them; ATM this is /usr/share/gocode/golang.xenproject.org/xenlight >>> Nit: This of course should have a `src/` between `gocode/` and `golang.xenproject.org/`. >>> >>> NB also that this naming scheme was designed so that at some point in the future, we could actually host the xenlight packages at the URL provided. >>> >>> -George >>> >> This new mechanism of modules is described here: >> https://golang.org/cmd/go/#hdr-Modules__module_versions__and_more >> >> The module system is intended to supersede the GOPATH approach and >> provide a way to get versioned dependencies, as such >> it does not rely on GOPATH at all and doesn't use sources or compiled >> packages present in GOPATH elements such as /usr/share/gocode >> and systematically fetch (at the asked version) and compile a copy of >> the dependency as it might be a different version from the one >> in GOPATH. >> >> As far as I tried, I have been unable to build my module even with the >> library installed. >> I have to use xenbits.xen.org/git-http/xen.git/tools/golang/xenlight (or >> one of its mirror) in order to build the module using the new >> mechanism (the golang.xenproject.org/xenlight works when building with >> modules mode disabled). > I took a look at the module stuff when it came out, and I was never able > to make sense of how it was supposed to work. Basically it is the same idea than a python virtualenv with |include-system-site-packages set to false: never use what is provided by the system and download everything in the exact version the manifest tells you to. | > <rant>On the whole, it seems they basically hate the idea of distro > packages, and seem intent on breaking them whenever people manage to > start to get them working.</rant> Actually yes because they don't want to be bound to the version provided by the distro (I will not enter the debate of whether it is a good thing or not) Nicolas
On 7/19/19 12:01 PM, George Dunlap wrote: > On 7/19/19 8:34 AM, Nicolas Belouin wrote: >> >> On 7/18/19 11:54 PM, George Dunlap wrote: >>> The Go bindings for libxl miss functions from libxl_utils, let's start >>> with the simple libxl_domid_to_name and its counterpart >>> libxl_name_to_domid. >>> >>> NB that C.GoString() will return "" if it's passed a NULL; see >>> https://github.com/golang/go/issues/32734#issuecomment-506835432 >>> >>> Signed-off-by: Nicolas Belouin <nicolas.belouin@gandi.net> >>> Signed-off-by: George Dunlap <george.dunlap@citrix.com> >>> --- >>> v3: >>> - Wire into build system >>> - Add reference to C.GoString() handling NULL to commit message >>> >>> Nicolas, could you test to see if this actually works for you? >> Tested it, it works. >> >> I must confess I do not use that import path as the new modules mechanism >> introduced in Go1.11 downloads and compile a versioned copy of every >> dependency per project, and this behavior is incompatible with the build >> system used here. > Also -- I can't check this in without somebody acking my changes; and > you're the most obvious person to do so. :-) > > -George I successfully tested your changes so consider them as acked. Nicolas
On 7/19/19 11:24 AM, Nicolas Belouin wrote: > > > On 7/19/19 12:09 PM, George Dunlap wrote: >> On 7/19/19 11:03 AM, Nicolas Belouin wrote: >>> >>> On 7/19/19 10:50 AM, George Dunlap wrote: >>>>> On Jul 19, 2019, at 9:47 AM, George Dunlap <george.dunlap@citrix.com> wrote: >>>>> >>>>> >>>>> >>>>>> On Jul 19, 2019, at 8:34 AM, Nicolas Belouin <nicolas.belouin@gandi.net> wrote: >>>>>> >>>>>> >>>>>> >>>>>> On 7/18/19 11:54 PM, George Dunlap wrote: >>>>>>> The Go bindings for libxl miss functions from libxl_utils, let's start >>>>>>> with the simple libxl_domid_to_name and its counterpart >>>>>>> libxl_name_to_domid. >>>>>>> >>>>>>> NB that C.GoString() will return "" if it's passed a NULL; see >>>>>>> https://github.com/golang/go/issues/32734#issuecomment-506835432 >>>>>>> >>>>>>> Signed-off-by: Nicolas Belouin <nicolas.belouin@gandi.net> >>>>>>> Signed-off-by: George Dunlap <george.dunlap@citrix.com> >>>>>>> --- >>>>>>> v3: >>>>>>> - Wire into build system >>>>>>> - Add reference to C.GoString() handling NULL to commit message >>>>>>> >>>>>>> Nicolas, could you test to see if this actually works for you? >>>>>> Tested it, it works. >>>>>> >>>>>> I must confess I do not use that import path as the new modules mechanism >>>>>> introduced in Go1.11 downloads and compile a versioned copy of every >>>>>> dependency per project, and this behavior is incompatible with the build >>>>>> system used here. >>>>> It’s possible that something fundamentally has changed, but I suspect that rather you don’t quite understand how the current build system is supposed to work. (In which case a write-up in the tree would probably be useful.) >>>>> >>>>> Go has always insisted that there be no binary compatibility between versions; so it’s always been necessary to re-compile all your libraries when upgrading from (say) 1.8 to 1.9. Which means that any useable distribution must also include all the source files necessary to recompile when you bump the version number. >>>>> >>>>> So the core mechanism of the “install” is actually to copy all the source files necessary into the right local directory such that the go compiler can find them; ATM this is /usr/share/gocode/golang.xenproject.org/xenlight >>>> Nit: This of course should have a `src/` between `gocode/` and `golang.xenproject.org/`. >>>> >>>> NB also that this naming scheme was designed so that at some point in the future, we could actually host the xenlight packages at the URL provided. >>>> >>>> -George >>>> >>> This new mechanism of modules is described here: >>> https://golang.org/cmd/go/#hdr-Modules__module_versions__and_more >>> >>> The module system is intended to supersede the GOPATH approach and >>> provide a way to get versioned dependencies, as such >>> it does not rely on GOPATH at all and doesn't use sources or compiled >>> packages present in GOPATH elements such as /usr/share/gocode >>> and systematically fetch (at the asked version) and compile a copy of >>> the dependency as it might be a different version from the one >>> in GOPATH. >>> >>> As far as I tried, I have been unable to build my module even with the >>> library installed. >>> I have to use xenbits.xen.org/git-http/xen.git/tools/golang/xenlight (or >>> one of its mirror) in order to build the module using the new >>> mechanism (the golang.xenproject.org/xenlight works when building with >>> modules mode disabled). >> I took a look at the module stuff when it came out, and I was never able >> to make sense of how it was supposed to work. > Basically it is the same idea than a python virtualenv with > |include-system-site-packages set to false: never use what is provided > by the system and download everything in the exact version the manifest > tells you to. > | >> <rant>On the whole, it seems they basically hate the idea of distro >> packages, and seem intent on breaking them whenever people manage to >> start to get them working.</rant> > Actually yes because they don't want to be bound to the version provided > by the distro (I will not enter the debate of whether it is a good thing > or not) If that's a requirement, the distro can provide multiple concurrent versions. There are lots of places where build systems aren't allowed to access the internet. And distro packages provides lots of useful things: discoverability, filtering (some level of review has been done to make sure this code us useful / safe), maintenance (local patches / fixes can be applied if upstream disappears), decentralization (code is still available even if upstream goes down / deletes his repositories). I like Go as a language, but in this particular aspect, the core developers just seem to be insane. -George
On 7/19/19 11:24 AM, Nicolas Belouin wrote: > > > On 7/19/19 12:09 PM, George Dunlap wrote: >> On 7/19/19 11:03 AM, Nicolas Belouin wrote: >>> >>> On 7/19/19 10:50 AM, George Dunlap wrote: >>>>> On Jul 19, 2019, at 9:47 AM, George Dunlap <george.dunlap@citrix.com> wrote: >>>>> >>>>> >>>>> >>>>>> On Jul 19, 2019, at 8:34 AM, Nicolas Belouin <nicolas.belouin@gandi.net> wrote: >>>>>> >>>>>> >>>>>> >>>>>> On 7/18/19 11:54 PM, George Dunlap wrote: >>>>>>> The Go bindings for libxl miss functions from libxl_utils, let's start >>>>>>> with the simple libxl_domid_to_name and its counterpart >>>>>>> libxl_name_to_domid. >>>>>>> >>>>>>> NB that C.GoString() will return "" if it's passed a NULL; see >>>>>>> https://github.com/golang/go/issues/32734#issuecomment-506835432 >>>>>>> >>>>>>> Signed-off-by: Nicolas Belouin <nicolas.belouin@gandi.net> >>>>>>> Signed-off-by: George Dunlap <george.dunlap@citrix.com> >>>>>>> --- >>>>>>> v3: >>>>>>> - Wire into build system >>>>>>> - Add reference to C.GoString() handling NULL to commit message >>>>>>> >>>>>>> Nicolas, could you test to see if this actually works for you? >>>>>> Tested it, it works. >>>>>> >>>>>> I must confess I do not use that import path as the new modules mechanism >>>>>> introduced in Go1.11 downloads and compile a versioned copy of every >>>>>> dependency per project, and this behavior is incompatible with the build >>>>>> system used here. >>>>> It’s possible that something fundamentally has changed, but I suspect that rather you don’t quite understand how the current build system is supposed to work. (In which case a write-up in the tree would probably be useful.) >>>>> >>>>> Go has always insisted that there be no binary compatibility between versions; so it’s always been necessary to re-compile all your libraries when upgrading from (say) 1.8 to 1.9. Which means that any useable distribution must also include all the source files necessary to recompile when you bump the version number. >>>>> >>>>> So the core mechanism of the “install” is actually to copy all the source files necessary into the right local directory such that the go compiler can find them; ATM this is /usr/share/gocode/golang.xenproject.org/xenlight >>>> Nit: This of course should have a `src/` between `gocode/` and `golang.xenproject.org/`. >>>> >>>> NB also that this naming scheme was designed so that at some point in the future, we could actually host the xenlight packages at the URL provided. >>>> >>>> -George >>>> >>> This new mechanism of modules is described here: >>> https://golang.org/cmd/go/#hdr-Modules__module_versions__and_more >>> >>> The module system is intended to supersede the GOPATH approach and >>> provide a way to get versioned dependencies, as such >>> it does not rely on GOPATH at all and doesn't use sources or compiled >>> packages present in GOPATH elements such as /usr/share/gocode >>> and systematically fetch (at the asked version) and compile a copy of >>> the dependency as it might be a different version from the one >>> in GOPATH. >>> >>> As far as I tried, I have been unable to build my module even with the >>> library installed. >>> I have to use xenbits.xen.org/git-http/xen.git/tools/golang/xenlight (or >>> one of its mirror) in order to build the module using the new >>> mechanism This will break as soon as we have support in golang/xenlight for libxl features not in the version of Xen you're using. E.g., suppose you're on Xen 4.12. Someone introduces a new feature in Xen 4.13, and plumbs it all the way from Xen, through libxl, *and* golang/xenlight. Now when *you* do a build, it will fail, because your version of golang will expect libxl features which your system doesn't have. I had always planned on getting golang.xenproject.org set up such that it could interact with the "normal" go get thing. If you want to help us figure out how to get that set up, that would be helpful. What would be *really* ideal is if we didn't have to link golang against one particular hypervisor. Maybe we need to use plugins? https://golang.org/pkg/plugin/ -George
On 7/19/19 1:04 PM, George Dunlap wrote: > On 7/19/19 11:24 AM, Nicolas Belouin wrote: >> >> On 7/19/19 12:09 PM, George Dunlap wrote: >>> On 7/19/19 11:03 AM, Nicolas Belouin wrote: >>>> On 7/19/19 10:50 AM, George Dunlap wrote: >>>>>> On Jul 19, 2019, at 9:47 AM, George Dunlap <george.dunlap@citrix.com> wrote: >>>>>> >>>>>> >>>>>> >>>>>>> On Jul 19, 2019, at 8:34 AM, Nicolas Belouin <nicolas.belouin@gandi.net> wrote: >>>>>>> >>>>>>> >>>>>>> >>>>>>> On 7/18/19 11:54 PM, George Dunlap wrote: >>>>>>>> The Go bindings for libxl miss functions from libxl_utils, let's start >>>>>>>> with the simple libxl_domid_to_name and its counterpart >>>>>>>> libxl_name_to_domid. >>>>>>>> >>>>>>>> NB that C.GoString() will return "" if it's passed a NULL; see >>>>>>>> https://github.com/golang/go/issues/32734#issuecomment-506835432 >>>>>>>> >>>>>>>> Signed-off-by: Nicolas Belouin <nicolas.belouin@gandi.net> >>>>>>>> Signed-off-by: George Dunlap <george.dunlap@citrix.com> >>>>>>>> --- >>>>>>>> v3: >>>>>>>> - Wire into build system >>>>>>>> - Add reference to C.GoString() handling NULL to commit message >>>>>>>> >>>>>>>> Nicolas, could you test to see if this actually works for you? >>>>>>> Tested it, it works. >>>>>>> >>>>>>> I must confess I do not use that import path as the new modules mechanism >>>>>>> introduced in Go1.11 downloads and compile a versioned copy of every >>>>>>> dependency per project, and this behavior is incompatible with the build >>>>>>> system used here. >>>>>> It’s possible that something fundamentally has changed, but I suspect that rather you don’t quite understand how the current build system is supposed to work. (In which case a write-up in the tree would probably be useful.) >>>>>> >>>>>> Go has always insisted that there be no binary compatibility between versions; so it’s always been necessary to re-compile all your libraries when upgrading from (say) 1.8 to 1.9. Which means that any useable distribution must also include all the source files necessary to recompile when you bump the version number. >>>>>> >>>>>> So the core mechanism of the “install” is actually to copy all the source files necessary into the right local directory such that the go compiler can find them; ATM this is /usr/share/gocode/golang.xenproject.org/xenlight >>>>> Nit: This of course should have a `src/` between `gocode/` and `golang.xenproject.org/`. >>>>> >>>>> NB also that this naming scheme was designed so that at some point in the future, we could actually host the xenlight packages at the URL provided. >>>>> >>>>> -George >>>>> >>>> This new mechanism of modules is described here: >>>> https://golang.org/cmd/go/#hdr-Modules__module_versions__and_more >>>> >>>> The module system is intended to supersede the GOPATH approach and >>>> provide a way to get versioned dependencies, as such >>>> it does not rely on GOPATH at all and doesn't use sources or compiled >>>> packages present in GOPATH elements such as /usr/share/gocode >>>> and systematically fetch (at the asked version) and compile a copy of >>>> the dependency as it might be a different version from the one >>>> in GOPATH. >>>> >>>> As far as I tried, I have been unable to build my module even with the >>>> library installed. >>>> I have to use xenbits.xen.org/git-http/xen.git/tools/golang/xenlight (or >>>> one of its mirror) in order to build the module using the new >>>> mechanism > This will break as soon as we have support in golang/xenlight for libxl > features not in the version of Xen you're using. > > E.g., suppose you're on Xen 4.12. Someone introduces a new feature in > Xen 4.13, and plumbs it all the way from Xen, through libxl, *and* > golang/xenlight. Now when *you* do a build, it will fail, because your > version of golang will expect libxl features which your system doesn't have. I know of that, and that can be overcome using modules as you can specify a branch version of the module you depends on (e.g you can set your dependency as being xxx/xenlight@stable-4.12). > > I had always planned on getting golang.xenproject.org set up such that > it could interact with the "normal" go get thing. If you want to help > us figure out how to get that set up, that would be helpful. As far as I looked into vanity URLs, you can't serve a subdirectory of a repository directly, but you can trick the system using a go-proxy. To do that you need two things. First, you need https://golang.xenproject.org/xenlight?go-get=1 <https://example.org/pkg/foo?go-get=1> to point to a page containing a <meta name="go-import" content="golang.xenproject.org mod https://golang.xenproject.org/moduleproxy <https://code.org/moduleproxy>"> And have golang.xenproject.org/moduleproxy to follow the specifications of module proxies by proposing '.zip' files containing the different versions of the module. This part for sure can be scripted to do the hard work of dynamically packaging the versions from the git repository on demand. > > What would be *really* ideal is if we didn't have to link golang against > one particular hypervisor. Maybe we need to use plugins? > https://golang.org/pkg/plugin/ This is the Go equivalent of ldopen, I'm not fond of this approach in general, but something like this can indeed do the trick to keep from linking against libxl directly, it would need a split in two modules to be easily usable though. Nicolas
On 7/19/19 2:19 PM, Nicolas Belouin wrote: > > > On 7/19/19 1:04 PM, George Dunlap wrote: >> On 7/19/19 11:24 AM, Nicolas Belouin wrote: >>> >>> On 7/19/19 12:09 PM, George Dunlap wrote: >>>> On 7/19/19 11:03 AM, Nicolas Belouin wrote: >>>>> On 7/19/19 10:50 AM, George Dunlap wrote: >>>>>>> On Jul 19, 2019, at 9:47 AM, George Dunlap <george.dunlap@citrix.com> wrote: >>>>>>> >>>>>>> >>>>>>> >>>>>>>> On Jul 19, 2019, at 8:34 AM, Nicolas Belouin <nicolas.belouin@gandi.net> wrote: >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> On 7/18/19 11:54 PM, George Dunlap wrote: >>>>>>>>> The Go bindings for libxl miss functions from libxl_utils, let's start >>>>>>>>> with the simple libxl_domid_to_name and its counterpart >>>>>>>>> libxl_name_to_domid. >>>>>>>>> >>>>>>>>> NB that C.GoString() will return "" if it's passed a NULL; see >>>>>>>>> https://github.com/golang/go/issues/32734#issuecomment-506835432 >>>>>>>>> >>>>>>>>> Signed-off-by: Nicolas Belouin <nicolas.belouin@gandi.net> >>>>>>>>> Signed-off-by: George Dunlap <george.dunlap@citrix.com> >>>>>>>>> --- >>>>>>>>> v3: >>>>>>>>> - Wire into build system >>>>>>>>> - Add reference to C.GoString() handling NULL to commit message >>>>>>>>> >>>>>>>>> Nicolas, could you test to see if this actually works for you? >>>>>>>> Tested it, it works. >>>>>>>> >>>>>>>> I must confess I do not use that import path as the new modules mechanism >>>>>>>> introduced in Go1.11 downloads and compile a versioned copy of every >>>>>>>> dependency per project, and this behavior is incompatible with the build >>>>>>>> system used here. >>>>>>> It’s possible that something fundamentally has changed, but I suspect that rather you don’t quite understand how the current build system is supposed to work. (In which case a write-up in the tree would probably be useful.) >>>>>>> >>>>>>> Go has always insisted that there be no binary compatibility between versions; so it’s always been necessary to re-compile all your libraries when upgrading from (say) 1.8 to 1.9. Which means that any useable distribution must also include all the source files necessary to recompile when you bump the version number. >>>>>>> >>>>>>> So the core mechanism of the “install” is actually to copy all the source files necessary into the right local directory such that the go compiler can find them; ATM this is /usr/share/gocode/golang.xenproject.org/xenlight >>>>>> Nit: This of course should have a `src/` between `gocode/` and `golang.xenproject.org/`. >>>>>> >>>>>> NB also that this naming scheme was designed so that at some point in the future, we could actually host the xenlight packages at the URL provided. >>>>>> >>>>>> -George >>>>>> >>>>> This new mechanism of modules is described here: >>>>> https://golang.org/cmd/go/#hdr-Modules__module_versions__and_more >>>>> >>>>> The module system is intended to supersede the GOPATH approach and >>>>> provide a way to get versioned dependencies, as such >>>>> it does not rely on GOPATH at all and doesn't use sources or compiled >>>>> packages present in GOPATH elements such as /usr/share/gocode >>>>> and systematically fetch (at the asked version) and compile a copy of >>>>> the dependency as it might be a different version from the one >>>>> in GOPATH. >>>>> >>>>> As far as I tried, I have been unable to build my module even with the >>>>> library installed. >>>>> I have to use xenbits.xen.org/git-http/xen.git/tools/golang/xenlight (or >>>>> one of its mirror) in order to build the module using the new >>>>> mechanism >> This will break as soon as we have support in golang/xenlight for libxl >> features not in the version of Xen you're using. >> >> E.g., suppose you're on Xen 4.12. Someone introduces a new feature in >> Xen 4.13, and plumbs it all the way from Xen, through libxl, *and* >> golang/xenlight. Now when *you* do a build, it will fail, because your >> version of golang will expect libxl features which your system doesn't have. > I know of that, and that can be overcome using modules as you can > specify a branch version of the module you depends on (e.g you can set > your dependency as being xxx/xenlight@stable-4.12). That's not terrible I guess. >> I had always planned on getting golang.xenproject.org set up such that >> it could interact with the "normal" go get thing. If you want to help >> us figure out how to get that set up, that would be helpful. > > As far as I looked into vanity URLs, you can't serve a subdirectory of a > repository directly, but you can trick the system using a go-proxy. > To do that you need two things. First, you need > https://golang.xenproject.org/xenlight?go-get=1 > <https://example.org/pkg/foo?go-get=1> to point to a page containing a > <meta name="go-import" content="golang.xenproject.org mod > https://golang.xenproject.org/moduleproxy <https://code.org/moduleproxy>"> > And have golang.xenproject.org/moduleproxy to follow the specifications > of module proxies by proposing '.zip' files containing the different > versions of the module. This part for sure can be scripted to do the > hard work of dynamically packaging the versions from the git repository > on demand. Yes, that's sort of what I had in mind -- although it might be better to statically update the content with a crontab or something, rather than regenerating it from scratch every connection. >> What would be *really* ideal is if we didn't have to link golang against >> one particular hypervisor. Maybe we need to use plugins? >> https://golang.org/pkg/plugin/ > > This is the Go equivalent of ldopen, I'm not fond of this approach in > general, but something like this can indeed do the trick to keep > from linking against libxl directly, it would need a split in two > modules to be easily usable though. It's certainly a lot of work, but it seems a lot nicer than having to recompile your application every time you upgrade your hypervisor. :-) But obviously all that requires work and testing, which requires someone who has enough time to make it happen. -George
diff --git a/tools/golang/xenlight/Makefile b/tools/golang/xenlight/Makefile index 0987305224..65923df59a 100644 --- a/tools/golang/xenlight/Makefile +++ b/tools/golang/xenlight/Makefile @@ -8,7 +8,7 @@ GOXL_PKG_DIR = /src/$(XEN_GOCODE_URL)/xenlight/ GOXL_INSTALL_DIR = $(GOCODE_DIR)$(GOXL_PKG_DIR) # PKGSOURCES: Files which comprise the distributed source package -PKGSOURCES = xenlight.go +PKGSOURCES = xenlight.go xenlight_utils.go GO ?= go diff --git a/tools/golang/xenlight/xenlight_utils.go b/tools/golang/xenlight/xenlight_utils.go new file mode 100644 index 0000000000..da1636842d --- /dev/null +++ b/tools/golang/xenlight/xenlight_utils.go @@ -0,0 +1,55 @@ +/* + * Copyright (C) 2019 Nicolas Belouin, Gandi SAS + * + * This library is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; + * version 2.1 of the License. + * + * This library is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with this library; If not, see <http://www.gnu.org/licenses/>. + */ +package xenlight + +/* +#cgo LDFLAGS: -lxenlight -lyajl -lxentoollog +#include <stdlib.h> +#include <libxl_utils.h> +*/ +import "C" + +import ( + "unsafe" +) + +//char* libxl_domid_to_name(libxl_ctx *ctx, uint32_t domid); +func (Ctx *Context) DomidToName(id Domid) (name string) { + cDomName := C.libxl_domid_to_name(Ctx.ctx, C.uint32_t(id)) + defer C.free(unsafe.Pointer(cDomName)) + + name = C.GoString(cDomName) + return +} + +//int libxl_name_to_domid(libxl_ct *ctx, const char *name, uint32_t *domid) +func (Ctx *Context) NameToDomid(name string) (id Domid, err error) { + cname := C.CString(name) + defer C.free(unsafe.Pointer(cname)) + + var cDomId C.uint32_t + + ret := C.libxl_name_to_domid(Ctx.ctx, cname, &cDomId) + if ret != 0 { + err = Error(-ret) + return + } + + id = Domid(cDomId) + + return +}