diff mbox

[2/3] Add a weekly coverity flight

Message ID 1454492776-23788-2-git-send-email-ian.campbell@citrix.com
State New, archived
Headers show

Commit Message

Ian Campbell Feb. 3, 2016, 9:46 a.m. UTC
This primarily consists of ts-coverity-scan and make-coverity-flight
which constructs the sole job.

The most recently scanned revision is pushed to a new coverity-scanned
branch in the usual xen.git, tests are run on the master branch.

For the cr-* integration we treat branch=coverity as a special case of
tree=xen. I didn't think tree=coverity made much sense, and would
probably reach tendrils into lots of other places (such as the
invocations of check_tested).

I initially thoughts that $c{CoverityEmail} would need to be an actual
account registered with scan, however a manual experiment using
email=security@xen.org was accepted by the service. An "analysis
complete" message was sent to security@ while individual results mails
were sent to each member of the coverity project who was configured to
receive them. I think this is what we want. The "analysis complete"
mail contained no sensitive data, but also no real information other
than "success" (or presumably "failure" if that were to be the case).
I think going to security@ is probably OK.

I have run this in non-uploading mode on the production infra and then
run the curl manually, adjusting the CLI until it works and updated
the script to match. I have not yet run in uploading mode but will do
so once another upload is allowed by the service,

In my experiments the curl command took ~35 minutes to complete (rate
in the 100-200k range). Not sure if this is a problem. Note that curl
is run on the controller (via system_checked) and consequently has no
timeout etc.

Note that the token must be supplied with </path/to/token and not
@/path/to/token. The latter appears to the server as a file upload
rather than a text field in a form which doesn't work. In early
attempts I thought that the trailing \n in /path/to/token might be an
issue and hence wrote a big comment. However having discovered < vs @
I am no longer 100% sure that is the case, but I left the comment
anyway since I can observe on the wire that the \n is included in the
upload (but each test takes ~35 mins and there is a ratelimit on the
server side too).

Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>
---
v2:
 - Split move of collect_xen_built_versions() into separate patch
 - Implemented support for coverity_upload = true (but don't yet set
   it)
 - Add host_hostflags to the job so it can actually run somewhere.
 - Call tsreadconfig() before referencing $r{coverity_upload} so that
   $r is actually populated.
 - use token=</path/to/token not token=@/path/to/token in curl
   arguments.
 - CoverityEmail == security@xen.org seems to work fine.
 - Add $flight to description.

Deployment notes:
 - Put cov-analysis-linux64-7.7.0.4.tar.gz in the Images
   directory. DONE in COLO
 - Populate $HOME/.xen-osstest/coverity-secret with the token
   DONE in COLO
 - Populate xen.git#coverity-scanned with an initial baseline, update
   ap-fetch-version-old to refer to it instead of master.
---
 ap-fetch-version     |   3 ++
 ap-fetch-version-old |   4 ++
 ap-print-url         |   2 +-
 ap-push              |   4 ++
 cr-daily-branch      |  18 ++++++++-
 cri-common           |   1 +
 crontab              |   1 +
 make-coverity-flight |  52 ++++++++++++++++++++++++
 production-config    |   4 ++
 sg-run-job           |   5 +++
 ts-coverity-scan     | 110 +++++++++++++++++++++++++++++++++++++++++++++++++++
 ts-xen-build-prep    |   2 +-
 12 files changed, 203 insertions(+), 3 deletions(-)
 create mode 100755 make-coverity-flight
 create mode 100755 ts-coverity-scan

Comments

Ian Campbell Feb. 3, 2016, 10:19 a.m. UTC | #1
On Wed, 2016-02-03 at 09:46 +0000, Ian Campbell wrote:
> [...]
> +sub build () {
> +    my $make = "make $makeflags";
> +
> +    # Pre build things we don't want coverity to scan, but which are
> +    # normally built by some other command.
> +    target_cmd_build($ho, 1000, $builddir, <<END);
> +cd $builddir/xen
> +./configure
> +$make -C tools/firmware/etherboot all
> +$make mini-os-dir
> +END
> +
> +    # Now the stuff we want coverity to look at
> +    target_cmd_build($ho, 9000, $builddir, <<END);
> +cd $builddir/xen
> +export PATH=$builddir/covtools/bin:\$PATH
> +cov-build --dir cov-int $make -C extras/mini-os/
> +cov-build --dir cov-int $make xen tools

This omits building stubdom, which Andy's original script also did.

However stubdom exists as a category in the scan webui and there have
previously been results for stubdoms.

Andy, I presume you deliberately started excluding stubdoms at some point?
I think this is probably the right thing to do, at least for now, since
stubdoms run with guest privileges so aren't hugely interesting, plus they
include an awful lot of third party code which we don't want to be
scanning+triaging (especially given how out of date some of the code is).

Ian.
Andrew Cooper Feb. 3, 2016, 10:46 a.m. UTC | #2
On 03/02/16 10:19, Ian Campbell wrote:
> On Wed, 2016-02-03 at 09:46 +0000, Ian Campbell wrote:
>>  [...]
>> +sub build () {
>> +    my $make = "make $makeflags";
>> +
>> +    # Pre build things we don't want coverity to scan, but which are
>> +    # normally built by some other command.
>> +    target_cmd_build($ho, 1000, $builddir, <<END);
>> +cd $builddir/xen
>> +./configure
>> +$make -C tools/firmware/etherboot all
>> +$make mini-os-dir
>> +END
>> +
>> +    # Now the stuff we want coverity to look at
>> +    target_cmd_build($ho, 9000, $builddir, <<END);
>> +cd $builddir/xen
>> +export PATH=$builddir/covtools/bin:\$PATH
>> +cov-build --dir cov-int $make -C extras/mini-os/
>> +cov-build --dir cov-int $make xen tools
> This omits building stubdom, which Andy's original script also did.
>
> However stubdom exists as a category in the scan webui and there have
> previously been results for stubdoms.
>
> Andy, I presume you deliberately started excluding stubdoms at some point?
> I think this is probably the right thing to do, at least for now, since
> stubdoms run with guest privileges so aren't hugely interesting, plus they
> include an awful lot of third party code which we don't want to be
> scanning+triaging (especially given how out of date some of the code is).

Correct.  That is precisely why I prevented stubdom and etherboot from
being scanned.

~Andrew
Ian Jackson Feb. 3, 2016, 12:12 p.m. UTC | #3
Ian Campbell writes ("[PATCH 2/3] Add a weekly coverity flight"):
> In my experiments the curl command took ~35 minutes to complete (rate
> in the 100-200k range). Not sure if this is a problem. Note that curl
> is run on the controller (via system_checked) and consequently has no
> timeout etc.

Can you use curl's builtin timeouts, or wrap it in alarm() ?
Otherwise I think it is possible for this to become wedged
indefinitely and need manual un-wedging.

AFAICT during the curl the only lock or resource which is held is the
build host share.  I think we can live with that.

> Note that the token must be supplied with </path/to/token and not
> @/path/to/token. The latter appears to the server as a file upload
> rather than a text field in a form which doesn't work. In early
> attempts I thought that the trailing \n in /path/to/token might be an
> issue and hence wrote a big comment. However having discovered < vs @
> I am no longer 100% sure that is the case, but I left the comment
> anyway since I can observe on the wire that the \n is included in the
> upload (but each test takes ~35 mins and there is a ratelimit on the
> server side too).

Right.

> Deployment notes:
>  - Put cov-analysis-linux64-7.7.0.4.tar.gz in the Images
>    directory. DONE in COLO
>  - Populate $HOME/.xen-osstest/coverity-secret with the token
>    DONE in COLO
>  - Populate xen.git#coverity-scanned with an initial baseline, update
>    ap-fetch-version-old to refer to it instead of master.
...
> +coverity)
> +	#XXX doesn't exist yet, use master for now repo_tree_rev_fetch_git xen $TREE_XEN coverity-scanned $LOCALREV_XEN

This XXX is out of date ?  And so the code should be fixed ?


> +	case $branch in
> +	    coverity)
> +		if [ "x$TREE_COVERITY" = x ]; then
> +		    export TREE_COVERITY=$TREE_XEN
> +		fi
> +		if [ "x$REVISION_COVERITY" = x ]; then
> +		    determine_version REVISION_COVERITY coverity COVERITY
> +		    export REVISION_COVERITY
> +		fi
> +		NEW_REVISION=$REVISION_COVERITY

I'm not sure why these variables are all REVISION_COVERITY rather than
REVISION_XEN.

But in general this looks good.

Ian.
Ian Jackson Feb. 3, 2016, 12:13 p.m. UTC | #4
Andrew Cooper writes ("Re: [PATCH 2/3] Add a weekly coverity flight"):
> On 03/02/16 10:19, Ian Campbell wrote:
> > Andy, I presume you deliberately started excluding stubdoms at some point?
> > I think this is probably the right thing to do, at least for now, since
> > stubdoms run with guest privileges so aren't hugely interesting, plus they
> > include an awful lot of third party code which we don't want to be
> > scanning+triaging (especially given how out of date some of the code is).
> 
> Correct.  That is precisely why I prevented stubdom and etherboot from
> being scanned.

I agree with this reasoning.

Ian.
Ian Campbell Feb. 3, 2016, 12:46 p.m. UTC | #5
On Wed, 2016-02-03 at 12:12 +0000, Ian Jackson wrote:
> Ian Campbell writes ("[PATCH 2/3] Add a weekly coverity flight"):
> > In my experiments the curl command took ~35 minutes to complete (rate
> > in the 100-200k range). Not sure if this is a problem. Note that curl
> > is run on the controller (via system_checked) and consequently has no
> > timeout etc.
> 
> Can you use curl's builtin timeouts, or wrap it in alarm() ?
> Otherwise I think it is possible for this to become wedged
> indefinitely and need manual un-wedging.

I added --max-time 3600.

> AFAICT during the curl the only lock or resource which is held is the
> build host share.  I think we can live with that.

There's the per-branch lock too, FWIW.

> > Deployment notes:
> >  - Put cov-analysis-linux64-7.7.0.4.tar.gz in the Images
> >    directory. DONE in COLO
> >  - Populate $HOME/.xen-osstest/coverity-secret with the token
> >    DONE in COLO
> >  - Populate xen.git#coverity-scanned with an initial baseline, update
> >    ap-fetch-version-old to refer to it instead of master.
> ...
> > +coverity)
> > +	#XXX doesn't exist yet, use master for now
> > repo_tree_rev_fetch_git xen $TREE_XEN coverity-scanned $LOCALREV_XEN
> 
> This XXX is out of date ?  And so the code should be fixed ?

coverity-scanned doesn't exist in xen.git yet, so for now this is still
correct, its the subject of the 3rd deployment note.

I'd be happy to push 9937763265d9597e5f2439249b16d995842cdf0f (the subject
of my recent adhoc test) to that new branch in xen.git right away though if
you agree and then update this code.

> > +	case $branch in
> > +	    coverity)
> > +		if [ "x$TREE_COVERITY" = x ]; then
> > +		    export TREE_COVERITY=$TREE_XEN
> > +		fi
> > +		if [ "x$REVISION_COVERITY" = x ]; then
> > +		    determine_version REVISION_COVERITY coverity
> > COVERITY
> > +		    export REVISION_COVERITY
> > +		fi
> > +		NEW_REVISION=$REVISION_COVERITY
> 
> I'm not sure why these variables are all REVISION_COVERITY rather than
> REVISION_XEN.

IIRC I had trouble getting cr-daily-branch's determine_version to populate
REVISION_XEN using values from "ap-fetch-version* coverity", maybe due to
the [ "x$tbranch" = "x$branch" ] ? 

or else there was some other wrinkle around that area.

Ian.
diff mbox

Patch

diff --git a/ap-fetch-version b/ap-fetch-version
index a7b658b..1e48b45 100755
--- a/ap-fetch-version
+++ b/ap-fetch-version
@@ -53,6 +53,9 @@  xen-4.*-testing)
 	repo_tree_rev_fetch_git xen \
 		$TREE_XEN staging-$branchcore $LOCALREV_XEN
 	;;
+coverity)
+	repo_tree_rev_fetch_git xen $TREE_XEN master $LOCALREV_XEN
+	;;
 qemu-mainline)
 	repo_tree_rev_fetch_git $branch \
 		$TREE_QEMU_MAINLINE master $LOCALREV_QEMU_UPSTREAM
diff --git a/ap-fetch-version-old b/ap-fetch-version-old
index e2c6b3b..9d6190f 100755
--- a/ap-fetch-version-old
+++ b/ap-fetch-version-old
@@ -59,6 +59,10 @@  xen-4.*-testing)
 	repo_tree_rev_fetch_git xen \
 		$TREE_XEN stable-$branchcore $LOCALREV_XEN
 	;;
+coverity)
+	#XXX doesn't exist yet, use master for now repo_tree_rev_fetch_git xen $TREE_XEN coverity-scanned $LOCALREV_XEN
+	repo_tree_rev_fetch_git xen $TREE_XEN master $LOCALREV_XEN
+	;;
 qemu-mainline)
         repo_tree_rev_fetch_git $branch \
 		$BASE_TREE_QEMU_UPSTREAM upstream-tested $LOCALREV_QEMU_UPSTREAM
diff --git a/ap-print-url b/ap-print-url
index 4088852..6ca000d 100755
--- a/ap-print-url
+++ b/ap-print-url
@@ -31,7 +31,7 @@  if info_linux_tree "$branch"; then
 fi
 
 case "$branch" in
-xen-*)
+xen-*|coverity)
         echo $TREE_XEN
 	;;
 qemu-mainline)
diff --git a/ap-push b/ap-push
index 8def652..97510c3 100755
--- a/ap-push
+++ b/ap-push
@@ -68,6 +68,10 @@  xen-*-testing)
 	xenversion=${xenversion#xen-}
 	git push $TREE_XEN $revision:refs/heads/stable-$xenversion
 	;;
+coverity)
+	cd $repos/xen
+	git push $TREE_XEN $revision:refs/heads/coverity-scanned
+	;;
 qemu-mainline)
 	cd $repos/qemu-mainline
 	git push $TREE_QEMU_UPSTREAM $revision:refs/heads/upstream-tested
diff --git a/cr-daily-branch b/cr-daily-branch
index 364238c..9594e18 100755
--- a/cr-daily-branch
+++ b/cr-daily-branch
@@ -205,7 +205,22 @@  fi
 case "$tree" in
 xen)
         realtree=$xenbranch
-	NEW_REVISION=$REVISION_XEN
+
+	case $branch in
+	    coverity)
+		if [ "x$TREE_COVERITY" = x ]; then
+		    export TREE_COVERITY=$TREE_XEN
+		fi
+		if [ "x$REVISION_COVERITY" = x ]; then
+		    determine_version REVISION_COVERITY coverity COVERITY
+		    export REVISION_COVERITY
+		fi
+		NEW_REVISION=$REVISION_COVERITY
+		;;
+	    *)
+		NEW_REVISION=$REVISION_XEN
+		;;
+	esac
 	;;
 linux)
         realtree=linux
@@ -259,6 +274,7 @@  fi
 
 case $branch in
 distros-*) makeflight=./make-distros-flight ;;
+coverity)  makeflight=./make-coverity-flight ;;
 *)         makeflight=./make-flight ;;
 esac
 
diff --git a/cri-common b/cri-common
index 6dfe8df..9f8bb0b 100644
--- a/cri-common
+++ b/cri-common
@@ -68,6 +68,7 @@  select_xenbranch () {
 	case "$branch" in
 	xen-unstable-smoke)	tree=xen;	xenbranch=$branch; qemuubranch=qemu-upstream-unstable;;
 	xen-*)			tree=xen;	xenbranch=$branch ;;
+	coverity)               tree=xen;       xenbranch=xen-unstable ;;
 	qemu-mainline)		tree=qemuu;	xenbranch=xen-unstable; qemuubranch=qemu-mainline;;
         qemu-upstream-*)    tree=qemuu; xenbranch=xen-${branch#qemu-upstream-};;
 	linux)			tree=linux;	xenbranch=xen-unstable ;;
diff --git a/crontab b/crontab
index 09b8d14..b6ced25 100755
--- a/crontab
+++ b/crontab
@@ -8,6 +8,7 @@  MAILTO=ian.jackson@citrix.com,ian.campbell@eu.citrix.com
 0		*	* * *		cd testing.git && BRANCHES=xen-unstable-smoke	./cr-for-branches branches -q "./cr-daily-branch --real"
 4-59/30		*	* * *		cd testing.git &&				./cr-for-branches branches -q "./cr-daily-branch --real"
 18		9	* * 1,3,5	cd testing.git && BRANCHES=linux-next		./cr-for-branches branches -w "./cr-daily-branch --real"
+18		9	* * 7		cd testing.git && BRANCHES=coverity		./cr-for-branches branches -w "./cr-daily-branch --real"
 18		4	* * *		cd testing.git && BRANCHES='linux-linus linux-mingo-tip-master linux-3.0 libvirt rumpuserxen' ./cr-for-branches branches -w "./cr-daily-branch --real"
 6-59/15   	*	* * *		cd testing.git && EXTRA_BRANCHES='linux-linus linux-3.0 rumpuserxen libvirt' ./cr-for-branches bisects -w "./cr-try-bisect --real"
 #8-59/5		*	* * *		cd bisects/adhoc.git &&	with-lock-ex -q data-tree-lock bash -c "./cr-try-bisect-adhoc; exit $?"
diff --git a/make-coverity-flight b/make-coverity-flight
new file mode 100755
index 0000000..dafb61c
--- /dev/null
+++ b/make-coverity-flight
@@ -0,0 +1,52 @@ 
+#!/bin/bash
+
+# This is part of "osstest", an automated testing framework for Xen.
+# Copyright (C) 2015 Citrix Inc.
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU Affero General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU Affero General Public License for more details.
+#
+# You should have received a copy of the GNU Affero General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+
+set -e -o posix
+
+branch=$1
+xenbranch=$2
+blessing=$3
+buildflight=$4
+
+flight=`./cs-flight-create $blessing $branch`
+
+. ./cri-common
+. ./ap-common
+. ./mfi-common
+
+defsuite=`getconfig DebianSuite`
+
+arch=amd64
+suite=$defsuite
+
+build_hostflags=share-build-$suite-$arch,arch-$arch,suite-$suite,purpose-build
+
+./cs-job-create $flight coverity-$arch coverity \
+	arch=$arch host_hostflags=$build_hostflags \
+	tree_xen=$TREE_COVERITY \
+	revision_xen=$REVISION_COVERITY \
+	coverity_upload=false
+
+echo $flight
+
+# Local variables:
+# mode: sh
+# sh-basic-offset: 2
+# indent-tabs-mode: nil
+# End:
diff --git a/production-config b/production-config
index f2f0584..e67a253 100644
--- a/production-config
+++ b/production-config
@@ -100,6 +100,10 @@  TftpGrubVersion XXXX-XX-XX
 XenUsePath /usr/groups/xencore/systems/bin/xenuse
 XenUseUser osstest
 
+# Results might include potential vulnerabilities.
+CoverityEmail security@xen.org
+CoverityTools cov-analysis-linux64-7.7.0.4.tar.gz
+
 # We use the IP address because Citrix can't manage reliable nameservice
 #DebianMirrorHost debian.uk.xensource.com
 #DebianMirrorHost 10.80.16.196
diff --git a/sg-run-job b/sg-run-job
index 20ebb64..7e592dd 100755
--- a/sg-run-job
+++ b/sg-run-job
@@ -445,6 +445,11 @@  proc prepare-build-host {} {
     run-ts . host-build-prep ts-xen-build-prep
 }
 
+proc need-hosts/coverity {} { return BUILD }
+proc run-job/coverity {} {
+    run-ts . = ts-coverity-scan + host
+}
+
 #---------- main program ----------
 
 jobdb::set-flight
diff --git a/ts-coverity-scan b/ts-coverity-scan
new file mode 100755
index 0000000..f3cc497
--- /dev/null
+++ b/ts-coverity-scan
@@ -0,0 +1,110 @@ 
+#!/usr/bin/perl -w
+# This is part of "osstest", an automated testing framework for Xen.
+# Copyright (C) 2015 Citrix Inc.
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU Affero General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU Affero General Public License for more details.
+#
+# You should have received a copy of the GNU Affero General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+use strict qw(vars);
+use DBI;
+use Osstest;
+use File::Path;
+use POSIX;
+use Osstest::TestSupport;
+use Osstest::BuildSupport;
+
+tsreadconfig();
+selectbuildhost(\@ARGV);
+# remaining arguments are passed as targets to "make"
+builddirsprops();
+
+# Require explicit opt in from flight construction
+my $coverity_upload = ($r{coverity_upload}//'false') =~ m/true/ ? 1 : 0;
+
+# This must contain exactly and only the token, for example there must
+# be no trailing "\n", otherwise it is included in the literal token,
+# which is then invalid.
+my $tokenfile = "$ENV{HOME}/.xen-osstest/coverity-secret";
+my $submit_url = "https://scan.coverity.com/builds?project=XenProject";
+
+sub checkout () {
+    prepbuilddirs();
+
+    build_clone($ho, 'xen', $builddir, 'xen');
+}
+
+sub covtools () {
+    target_putfile($ho, 100, "$c{Images}/$c{CoverityTools}", "$builddir/covtools.tar.gz");
+    target_cmd($ho, <<END, 100);
+set -xe
+c=$builddir/covtools
+mkdir -p \$c
+cd \$c
+tar --strip-components=1 -xaf $builddir/covtools.tar.gz
+END
+}
+
+sub build () {
+    my $make = "make $makeflags";
+
+    # Pre build things we don't want coverity to scan, but which are
+    # normally built by some other command.
+    target_cmd_build($ho, 1000, $builddir, <<END);
+cd $builddir/xen
+./configure
+$make -C tools/firmware/etherboot all
+$make mini-os-dir
+END
+
+    # Now the stuff we want coverity to look at
+    target_cmd_build($ho, 9000, $builddir, <<END);
+cd $builddir/xen
+export PATH=$builddir/covtools/bin:\$PATH
+cov-build --dir cov-int $make -C extras/mini-os/
+cov-build --dir cov-int $make xen tools
+
+tar czvf xen-coverity.tgz cov-int
+END
+
+    built_stash_file($ho, $builddir,
+		     "xen-coverity.tgz", "xen/xen-coverity.tgz", 0);
+}
+
+sub upload() {
+    my $xen_version = target_cmd_output($ho, <<END, 30);
+    cd $builddir/xen
+    make xenversion
+END
+
+    my @form_args;
+    push @form_args, "token=\<$tokenfile";
+    push @form_args, "email=$c{CoverityEmail}";
+    push @form_args, "file=\@$stash/build/xen-coverity.tgz";
+    push @form_args, "version=$xen_version";
+    push @form_args, "description=$flight: $r{tree_xen} $r{built_revision_xen}";
+
+    my @args = map { ("--form", $_) } @form_args;
+    push @args, $submit_url;
+
+    if ($coverity_upload) {
+	system_checked("curl", @args);
+    } else {
+	logm("Not uploading: curl args: ".(join " ", map { qq("$_") } @args));
+    }
+}
+
+checkout();
+covtools();
+build();
+collect_xen_built_versions();
+upload();
diff --git a/ts-xen-build-prep b/ts-xen-build-prep
index b35e91b..c2383db 100755
--- a/ts-xen-build-prep
+++ b/ts-xen-build-prep
@@ -206,7 +206,7 @@  sub prep () {
                       autoconf automake libtool xsltproc
                       libxml2-utils libxml2-dev
                       libdevmapper-dev w3c-dtd-xhtml libxml-xpath-perl
-                      ccache nasm checkpolicy ebtables);
+                      ccache nasm checkpolicy ebtables curl);
 
     if ($ho->{Suite} =~ m/wheezy|squeeze|lenny/) {
 	push(@packages, "libnl-dev");