Coveralls dropped suddenly

[CasperWA]

It seems that since PR #3305 coveralls has dropped to 20-30 % (from the original ~75 %).

Looking at this build versus the one immediately prior to it it seems coveralls now also takes into account the absolute file paths on the Travis builds (see under TREE under SOURCE FILES ...).
It seems to be when testing sphinxext, since this subfolder has a high coverage under this tree - and this is also what was touched by the PR.

[giovannipizzi]

Indeed, sphinxes went to zero, but I think the other big problem a you say is that it now considers the / path and the python venvs as well...
The reason is, I think, that apparently now some things are seen by coverall not coming from the local tree, but from its absolute path (in the venv folder), so it splits the results in two groups, both with lower coverage (if you open up the / tree and go down, you see that in there you get back the full coverage of the sphinxext).

This is clearly due to the main changes in the PR (i.e. changing the sys.path).

@ltalirz any suggestion on this?

[giovannipizzi]

It's possibly due to the way we run coverage, that for some tests it reports relative paths and for others absolute paths.
Probably this line:

aiida-core/.ci/test_script.sh

Line 42 in b0dd177

pytest --cov aiida --cov-append --cov-report= -vv aiida/sphinxext/tests

This might need to specify which PYTHONPATH to use so it sees files as relative paths.
I guess this is due to a missing specification of the package (--source in the coverage executable), that with pytest should be specified with --cov-config=.coveragerc? I'll try to check this.

Note also that some other files appear with an absolute path - possibly because of the same test, or maybe of some of the other coverage lines? To be checked.