code coverage and teuthology
diff mbox

Message ID 50EF49C4.1020909@dachary.org
State New, archived
Headers show

Commit Message

Loic Dachary Jan. 10, 2013, 11:07 p.m. UTC
Hi,

I successfully run teuthology with the proposed 3node_rgw.yaml [1] and changing the flavor from basic to gcov [2]. I hoped to use cov-init.sh ( https://github.com/ceph/teuthology/blob/master/coverage/cov-init.sh ) and then coverage.sh but I can't figure out how to make it work.

Hints would be most appreciated ;-)

[1] 3node_rgw.yaml

check-locks: false
interactive-on-error: true
overrides:
  ceph:
    branch: master
    fs: xfs
roles:
- - mon.a
  - mon.c
  - osd.0
- - mon.b
  - mds.a
  - osd.1
- - client.0
tasks:
- ceph: null
- rgw:
  - client.0
- interactive:

targets:
(snip)

[2] http://gitbuilder.ceph.com/ceph-tarball-precise-x86_64-gcov/

Comments

Josh Durgin Jan. 14, 2013, 5:26 p.m. UTC | #1
On 01/10/2013 03:07 PM, Loic Dachary wrote:
> Hi,
>
> I successfully run teuthology with the proposed 3node_rgw.yaml [1] and changing the flavor from basic to gcov [2]. I hoped to use cov-init.sh ( https://github.com/ceph/teuthology/blob/master/coverage/cov-init.sh ) and then coverage.sh but I can't figure out how to make it work.
>
> Hints would be most appreciated ;-)

Looking at how it's run automatically might help:

https://github.com/ceph/teuthology/blob/master/teuthology/coverage.py#L88

> [1] 3node_rgw.yaml
>
> check-locks: false
> interactive-on-error: true
> overrides:
>    ceph:
>      branch: master
>      fs: xfs

You should also add 'coverage: true' for the ceph task overrides.
This way daemons are killed with SIGTERM, and the atexit function
that outputs coverage information will run.

Then you don't need your patch changing the flavor either.
For each task X, the docstring for teuthology.task.X.task documents
example usage and extra options like this.

Josh

> roles:
> - - mon.a
>    - mon.c
>    - osd.0
> - - mon.b
>    - mds.a
>    - osd.1
> - - client.0
> tasks:
> - ceph: null
> - rgw:
>    - client.0
> - interactive:
>
> targets:
> (snip)
>
> [2] http://gitbuilder.ceph.com/ceph-tarball-precise-x86_64-gcov/
>
> diff --git a/teuthology/task/ceph.py b/teuthology/task/ceph.py
> index b62bc1d..99d4f18 100644
> --- a/teuthology/task/ceph.py
> +++ b/teuthology/task/ceph.py
> @@ -1027,7 +1027,7 @@ def task(ctx, config):
>       dist = 'precise'
>       format = 'tarball'
>       arch = 'x86_64'
> -    flavor = 'basic'
> +    flavor = 'gcov'
>
>       # First element: controlled by user (or not there, by default):
>       # used to choose the right distribution, e.g. "oneiric".
> diff --git a/teuthology/task/kernel.py b/teuthology/task/kernel.py
> index 5c34433..54eaebb 100644
> --- a/teuthology/task/kernel.py
> +++ b/teuthology/task/kernel.py
> @@ -183,7 +183,7 @@ def download_deb(ctx, config):
>                   package='kernel',
>                   sha1=src,
>                   format='deb',
> -                flavor='basic',
> +                flavor='gcov',
>                   arch='x86_64',
>                   dist='precise',
>                   )
> @@ -408,7 +408,7 @@ def task(ctx, config):
>                   branch=role_config.get('branch'),
>                   tag=role_config.get('tag'),
>                   sha1=role_config.get('sha1'),
> -                flavor='basic',
> +                flavor='gcov',
>                   format='deb',
>                   dist='precise',
>                   arch='x86_64',
>

--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Loic Dachary Jan. 15, 2013, 10:10 a.m. UTC | #2
On 01/14/2013 06:26 PM, Josh Durgin wrote:
>
> Looking at how it's run automatically might help:
>
> https://github.com/ceph/teuthology/blob/master/teuthology/coverage.py#L88
>
>
> You should also add 'coverage: true' for the ceph task overrides.
> This way daemons are killed with SIGTERM, and the atexit function
> that outputs coverage information will run.
>
> Then you don't need your patch changing the flavor either.
> For each task X, the docstring for teuthology.task.X.task documents
> example usage and extra options like this.
Hi,

That helped a lot, thanks :-) I think I'm almost there. After running:

./virtualenv/bin/teuthology --archive /tmp/a1 /srv/3node_rgw.yaml

wget -O /tmp/build/tmp.tgz http://gitbuilder.ceph.com/ceph-tarball-precise-x86_64-gcov/sha1/$(cat /tmp/a1/ceph-sha1)/ceph.x86_64.tgz

echo ceph_build_output_dir: /tmp/build >> ~/.teuthology.yaml

./virtualenv/bin/teuthology-coverage -v --html-output /tmp/html --lcov-output /tmp/lcov --cov-tools-dir /srv/teuthology/coverage /tmp

I get

INFO:teuthology.coverage:initializing coverage data...
Retrieving source and .gcno files...
Initializing lcov files...
Deleting all .da files in /tmp/lcov/ceph/src and subdirectories
Done.
Capturing coverage data from /tmp/lcov/ceph/src
Found gcov version: 4.7.2
Scanning /tmp/lcov/ceph/src for .gcno files ...
Found 692 graph files in /tmp/lcov/ceph/src
Processing src/test_libhadoopcephfs_build-AuthMethodList.gcno
geninfo: ERROR: /tmp/lcov/ceph/src/test_libhadoopcephfs_build-AuthMethodList.gcno: reached unexpected end of file

root@ceph:/srv/teuthology# ls -l /tmp/lcov/ceph/src/test_libhadoopcephfs_build-AuthMethodList.gcno
-rw-r--r-- 1 root root 41088 Jan 15 09:49 /tmp/lcov/ceph/src/test_libhadoopcephfs_build-AuthMethodList.gcno

I'm using

lcov: LCOV version 1.9

The only problem I can think of is that the machine I'm running lcov on is a Debian GNU/Linux Wheezy, trying to analyze coverage for binaries created for Ubuntu Precise. They are both amd64 but .gcno files may have dependencies to the toolchain.

Did you ever run into similar problems ?

Cheers

-----------------------------
check-locks: false
interactive-on-error: true
overrides:
  ceph:
    branch: master
    flavor: gcov
    coverage: true
    fs: xfs
roles:
- - mon.a
  - mon.c
  - osd.0
- - mon.b
  - mds.a
  - osd.1
- - client.0
tasks:
- ceph:
   coverage: true

targets:
...
Josh Durgin Jan. 15, 2013, 5:21 p.m. UTC | #3
On 01/15/2013 02:10 AM, Loic Dachary wrote:
> On 01/14/2013 06:26 PM, Josh Durgin wrote:
>>
>> Looking at how it's run automatically might help:
>>
>> https://github.com/ceph/teuthology/blob/master/teuthology/coverage.py#L88
>>
>>
>> You should also add 'coverage: true' for the ceph task overrides.
>> This way daemons are killed with SIGTERM, and the atexit function
>> that outputs coverage information will run.
>>
>> Then you don't need your patch changing the flavor either.
>> For each task X, the docstring for teuthology.task.X.task documents
>> example usage and extra options like this.
> Hi,
>
> That helped a lot, thanks :-) I think I'm almost there. After running:
>
> ./virtualenv/bin/teuthology --archive /tmp/a1 /srv/3node_rgw.yaml
>
> wget -O /tmp/build/tmp.tgz http://gitbuilder.ceph.com/ceph-tarball-precise-x86_64-gcov/sha1/$(cat /tmp/a1/ceph-sha1)/ceph.x86_64.tgz
>
> echo ceph_build_output_dir: /tmp/build >> ~/.teuthology.yaml
>
> ./virtualenv/bin/teuthology-coverage -v --html-output /tmp/html --lcov-output /tmp/lcov --cov-tools-dir /srv/teuthology/coverage /tmp
>
> I get
>
> INFO:teuthology.coverage:initializing coverage data...
> Retrieving source and .gcno files...
> Initializing lcov files...
> Deleting all .da files in /tmp/lcov/ceph/src and subdirectories
> Done.
> Capturing coverage data from /tmp/lcov/ceph/src
> Found gcov version: 4.7.2
> Scanning /tmp/lcov/ceph/src for .gcno files ...
> Found 692 graph files in /tmp/lcov/ceph/src
> Processing src/test_libhadoopcephfs_build-AuthMethodList.gcno
> geninfo: ERROR: /tmp/lcov/ceph/src/test_libhadoopcephfs_build-AuthMethodList.gcno: reached unexpected end of file
>
> root@ceph:/srv/teuthology# ls -l /tmp/lcov/ceph/src/test_libhadoopcephfs_build-AuthMethodList.gcno
> -rw-r--r-- 1 root root 41088 Jan 15 09:49 /tmp/lcov/ceph/src/test_libhadoopcephfs_build-AuthMethodList.gcno
>
> I'm using
>
> lcov: LCOV version 1.9
>
> The only problem I can think of is that the machine I'm running lcov on is a Debian GNU/Linux Wheezy, trying to analyze coverage for binaries created for Ubuntu Precise. They are both amd64 but .gcno files may have dependencies to the toolchain.
>
> Did you ever run into similar problems ?

I think I did when I built and ran on debian, and it was fixed with a
later version of lcov (I think 1.9-2). I didn't try doing the coverage
analysis on a different distribution from where ceph was built and run
though, so that may also cause some issues.

Josh
--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Dan Mick Jan. 15, 2013, 8:23 p.m. UTC | #4
It would not surprise me at all if gcov files are *highly* version 
dependent.  I don't know one way or the other, but it seems very possible.

On 01/15/2013 09:21 AM, Josh Durgin wrote:
> On 01/15/2013 02:10 AM, Loic Dachary wrote:
>> On 01/14/2013 06:26 PM, Josh Durgin wrote:
>>>
>>> Looking at how it's run automatically might help:
>>>
>>> https://github.com/ceph/teuthology/blob/master/teuthology/coverage.py#L88
>>>
>>>
>>>
>>> You should also add 'coverage: true' for the ceph task overrides.
>>> This way daemons are killed with SIGTERM, and the atexit function
>>> that outputs coverage information will run.
>>>
>>> Then you don't need your patch changing the flavor either.
>>> For each task X, the docstring for teuthology.task.X.task documents
>>> example usage and extra options like this.
>> Hi,
>>
>> That helped a lot, thanks :-) I think I'm almost there. After running:
>>
>> ./virtualenv/bin/teuthology --archive /tmp/a1 /srv/3node_rgw.yaml
>>
>> wget -O /tmp/build/tmp.tgz
>> http://gitbuilder.ceph.com/ceph-tarball-precise-x86_64-gcov/sha1/$(cat
>> /tmp/a1/ceph-sha1)/ceph.x86_64.tgz
>>
>> echo ceph_build_output_dir: /tmp/build >> ~/.teuthology.yaml
>>
>> ./virtualenv/bin/teuthology-coverage -v --html-output /tmp/html
>> --lcov-output /tmp/lcov --cov-tools-dir /srv/teuthology/coverage /tmp
>>
>> I get
>>
>> INFO:teuthology.coverage:initializing coverage data...
>> Retrieving source and .gcno files...
>> Initializing lcov files...
>> Deleting all .da files in /tmp/lcov/ceph/src and subdirectories
>> Done.
>> Capturing coverage data from /tmp/lcov/ceph/src
>> Found gcov version: 4.7.2
>> Scanning /tmp/lcov/ceph/src for .gcno files ...
>> Found 692 graph files in /tmp/lcov/ceph/src
>> Processing src/test_libhadoopcephfs_build-AuthMethodList.gcno
>> geninfo: ERROR:
>> /tmp/lcov/ceph/src/test_libhadoopcephfs_build-AuthMethodList.gcno:
>> reached unexpected end of file
>>
>> root@ceph:/srv/teuthology# ls -l
>> /tmp/lcov/ceph/src/test_libhadoopcephfs_build-AuthMethodList.gcno
>> -rw-r--r-- 1 root root 41088 Jan 15 09:49
>> /tmp/lcov/ceph/src/test_libhadoopcephfs_build-AuthMethodList.gcno
>>
>> I'm using
>>
>> lcov: LCOV version 1.9
>>
>> The only problem I can think of is that the machine I'm running lcov
>> on is a Debian GNU/Linux Wheezy, trying to analyze coverage for
>> binaries created for Ubuntu Precise. They are both amd64 but .gcno
>> files may have dependencies to the toolchain.
>>
>> Did you ever run into similar problems ?
>
> I think I did when I built and ran on debian, and it was fixed with a
> later version of lcov (I think 1.9-2). I didn't try doing the coverage
> analysis on a different distribution from where ceph was built and run
> though, so that may also cause some issues.
>
> Josh
> --
> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Loic Dachary Jan. 16, 2013, 12:52 p.m. UTC | #5
On 01/15/2013 06:21 PM, Josh Durgin wrote:
> On 01/15/2013 02:10 AM, Loic Dachary wrote:
>> On 01/14/2013 06:26 PM, Josh Durgin wrote:
>>>
>>> Looking at how it's run automatically might help:
>>>
>>> https://github.com/ceph/teuthology/blob/master/teuthology/coverage.py#L88
>>>
>>>
>>> You should also add 'coverage: true' for the ceph task overrides.
>>> This way daemons are killed with SIGTERM, and the atexit function
>>> that outputs coverage information will run.
>>>
>>> Then you don't need your patch changing the flavor either.
>>> For each task X, the docstring for teuthology.task.X.task documents
>>> example usage and extra options like this.
>> Hi,
>>
>> That helped a lot, thanks :-) I think I'm almost there. After running:
>>
>> ./virtualenv/bin/teuthology --archive /tmp/a1 /srv/3node_rgw.yaml
>>
>> wget -O /tmp/build/tmp.tgz http://gitbuilder.ceph.com/ceph-tarball-precise-x86_64-gcov/sha1/$(cat /tmp/a1/ceph-sha1)/ceph.x86_64.tgz
>>
>> echo ceph_build_output_dir: /tmp/build >> ~/.teuthology.yaml
>>
>> ./virtualenv/bin/teuthology-coverage -v --html-output /tmp/html --lcov-output /tmp/lcov --cov-tools-dir /srv/teuthology/coverage /tmp
>>
>> I get
>>
>> INFO:teuthology.coverage:initializing coverage data...
>> Retrieving source and .gcno files...
>> Initializing lcov files...
>> Deleting all .da files in /tmp/lcov/ceph/src and subdirectories
>> Done.
>> Capturing coverage data from /tmp/lcov/ceph/src
>> Found gcov version: 4.7.2
>> Scanning /tmp/lcov/ceph/src for .gcno files ...
>> Found 692 graph files in /tmp/lcov/ceph/src
>> Processing src/test_libhadoopcephfs_build-AuthMethodList.gcno
>> geninfo: ERROR: /tmp/lcov/ceph/src/test_libhadoopcephfs_build-AuthMethodList.gcno: reached unexpected end of file
>>
>> root@ceph:/srv/teuthology# ls -l /tmp/lcov/ceph/src/test_libhadoopcephfs_build-AuthMethodList.gcno
>> -rw-r--r-- 1 root root 41088 Jan 15 09:49 /tmp/lcov/ceph/src/test_libhadoopcephfs_build-AuthMethodList.gcno
>>
>> I'm using
>>
>> lcov: LCOV version 1.9
>>
>> The only problem I can think of is that the machine I'm running lcov on is a Debian GNU/Linux Wheezy, trying to analyze coverage for binaries created for Ubuntu Precise. They are both amd64 but .gcno files may have dependencies to the toolchain.
>>
>> Did you ever run into similar problems ?
>
> I think I did when I built and ran on debian, and it was fixed with a
> later version of lcov (I think 1.9-2). I didn't try doing the coverage
> analysis on a different distribution from where ceph was built and run
> though, so that may also cause some issues.
It was indeed a compatibility problem : running lcov on precise works fine.

Thanks :-)
>
> Josh
> -- 
> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Patch
diff mbox

diff --git a/teuthology/task/ceph.py b/teuthology/task/ceph.py
index b62bc1d..99d4f18 100644
--- a/teuthology/task/ceph.py
+++ b/teuthology/task/ceph.py
@@ -1027,7 +1027,7 @@  def task(ctx, config):
     dist = 'precise'
     format = 'tarball'
     arch = 'x86_64'
-    flavor = 'basic'
+    flavor = 'gcov'

     # First element: controlled by user (or not there, by default):
     # used to choose the right distribution, e.g. "oneiric".
diff --git a/teuthology/task/kernel.py b/teuthology/task/kernel.py
index 5c34433..54eaebb 100644
--- a/teuthology/task/kernel.py
+++ b/teuthology/task/kernel.py
@@ -183,7 +183,7 @@  def download_deb(ctx, config):
                 package='kernel',
                 sha1=src,
                 format='deb',
-                flavor='basic',
+                flavor='gcov',
                 arch='x86_64',
                 dist='precise',
                 )
@@ -408,7 +408,7 @@  def task(ctx, config):
                 branch=role_config.get('branch'),
                 tag=role_config.get('tag'),
                 sha1=role_config.get('sha1'),
-                flavor='basic',
+                flavor='gcov',
                 format='deb',
                 dist='precise',
                 arch='x86_64',