Make use of PHPUnit 9 code coverage and reporting #1431

BlairCooper · 2020-11-13T01:06:55Z

Added support for Clover HTML report.
Pass phpunit.xml configuration values to filter and code coverage
classes.

Fixes #1430 Added support for Clover HTML report. Pass phpunit.xml configuration values to filter and code coverage classes.

… into phpunit9coverage

codecov · 2020-11-13T01:46:34Z

Codecov Report

Merging #1431 (29eba89) into master (e5b487f) will decrease coverage by 0.19%.
The diff coverage is 38.81%.

@@             Coverage Diff              @@
##             master    #1431      +/-   ##
============================================
- Coverage     54.27%   54.08%   -0.20%     
- Complexity     9119     9178      +59     
============================================
  Files           467      469       +2     
  Lines         22153    22289     +136     
============================================
+ Hits          12024    12054      +30     
- Misses        10129    10235     +106

Impacted Files	Coverage Δ	Complexity Δ
...t/formatter9/CloverHtmlPHPUnitResultFormatter9.php	`0.00% <0.00%> (ø)`	`4.00 <4.00> (?)`
classes/phing/tasks/ext/phpunit/PHPUnitTask.php	`52.06% <23.52%> (+8.14%)`	`99.00 <0.00> (+13.00)`
...asses/phing/tasks/ext/phpunit/FormatterElement.php	`45.45% <33.33%> (-0.89%)`	`20.00 <0.00> (+1.00)`	⬇️
...ses/phing/tasks/ext/phpunit/PHPUnitTestRunner9.php	`48.54% <48.54%> (ø)`	`47.00 <47.00> (?)`
...ses/phing/tasks/ext/phpunit/PHPUnitTestRunner8.php	`0.00% <0.00%> (-43.45%)`	`54.00% <0.00%> (-6.00%)`

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update e5b487f...29eba89. Read the comment docs.

siad007 · 2020-11-14T12:32:06Z

test/build.xml

@@ -51,28 +51,13 @@
    </target>

    <target name="reports" depends="configure">
-        <delete dir="tmp"/>
+	<delete dir="tmp"/>


Please revert this ws change.

siad007 · 2020-11-14T12:33:11Z

classes/phing/tasks/ext/phpunit/PHPUnitTask.php

@@ -1,4 +1,6 @@
 <?php
+use PHPUnit\TextUI\XmlConfiguration\CodeCoverage\CodeCoverage;


Imports should be right after the file comment.

Unused, removed.

siad007 · 2020-11-14T12:36:54Z

classes/phing/tasks/ext/phpunit/FormatterElement.php

@@ -168,6 +168,9 @@ public function getFormatter()
            $this->formatter = new SummaryPHPUnitResultFormatter7($this->parent);
        } elseif ($this->type === "clover") {
            $this->formatter = new CloverPHPUnitResultFormatter7($this->parent);
+        } elseif ($this->type === "clover-html") {


Should be documented in https://github.com/phingofficial/guide/blob/d1c4b97b72a6ff9cba1801406d871dcfb2aaedcd/source/appendixes/optionaltasks.xml#L8905-L8915

I have added to the documentation as part of a separate change to follow this pull request.

siad007 · 2020-11-14T12:53:07Z

classes/phing/tasks/ext/phpunit/PHPUnitTestRunner9.php

+    /**
+     * @var \PHPUnit\Runner\TestHook[]
+     */
+    private $listeners = [];


We should rename all occurences to hook(s) instead to cover the new behavior of phpunit.
Same for all accessors names as well.

Put \PHPUnit\Framework\TestListener back on the class. Changing to use TestHook is beyond the scope of this change. It is certainly something that should be looked at in the future.

siad007 · 2020-11-14T12:54:59Z

test/classes/phing/tasks/ext/PHPUnitTaskTest.php

@@ -51,6 +51,33 @@ public function testXmlFormatter()
        $this->assertInLogs("<testcase name=\"testSayHello\" class=\"HelloWorldTest\"");
    }

+    /**
+     *
+     */


Please remove useless empty comment blocks.

siad007 · 2020-11-14T12:57:54Z

classes/phing/tasks/ext/phpunit/PHPUnitTestRunner9.php

+     */
+    public function __construct(
+        Project $project,
+        $groups = [],


Please add missing argument type hints.
Same for all new methods in this PR.

siad007

I need some more time to go through this PR.
So the request changes could be extended later on.

siad007 · 2020-11-14T13:04:45Z

classes/phing/tasks/ext/phpunit/PHPUnitTestRunner9.php

+
+        if ($res->riskyCount() > 0) {
+            $this->hasRisky = true;
+        }


Please reduce complexity and avoid not needed conditions.

$this->hasFailures = $res->failureCount() > 0;

As seen in the Codecov Report

Complexity 9119 9187 +68

There is also a big raise on missing test coverage for your changes:

Misses 10129 10242 +113

We should investigate why.

Change the complexity suggestion. The raise in missing coverage is likely due to adding PHPUnitTestRunner9 which results in PHPUnitTestRunner8 not being used.

siad007 · 2020-11-14T14:10:54Z

classes/phing/tasks/ext/phpunit/PHPUnitTestRunner9.php

+ * @author  Michiel Rook <mrook@php.net>
+ * @package phing.tasks.ext.phpunit
+ */
+class PHPUnitTestRunner9 implements \PHPUnit\Runner\TestHook


Please reenable the required and useful enhancement for collecting coverage data by db.
Without this feature we are not able to merge coverage across different runs of phpunit tasks to merge test coverage from multiple test runs from different projects.

That isn't something I removed. It appears PHPUnit no longer writes to the coverage database.

The code coverage merge feature (special the db function) has nothing to do with phpunit itself.
Please have a look at the coverage tasks https://github.com/phingofficial/task-coverage/tree/master/src

siad007 · 2020-11-14T15:04:12Z

test/etc/tasks/ext/phpunit/build.xml

+    </target>
+
+    <target name="testPhpunitConfig">
+        <phpunit configuration="phpunit-test.xml" codecoverage="false" >


This will increase test execution time in build from 35s to 61s.

True, but also adds to the code coverage and specifically tests new code that was added.

but also adds to the code coverage and specifically tests new code that was added.

To be honest - it just executes the whole test suite again and ignores code-coverage entirely! (codecoverage="false")

siad007 · 2020-11-14T20:32:26Z

@BlairCooper as you can see in #1432, there is no problem with the merge coverage feature.
Please revert all changes, which are not part of CloverHtmlPHPUnitResultFormatter9 and FormatterElement
Also the extended configuration feature with PHPUnitTask::codeCoverageConfig is a great new addition, which should go into the next release.

BlairCooper · 2020-11-14T20:54:18Z

Reworking with just new formatter and runner for PHPUnit 9.

BlairCooper added 3 commits November 12, 2020 16:39

Make use of PHPUnit 9 code coverage and reporting

8180e2c

Fixes #1430 Added support for Clover HTML report. Pass phpunit.xml configuration values to filter and code coverage classes.

Make use of PHPUnit 9 code coverage and reporting

4ac5089

Fixes #1430 Added support for Clover HTML report. Pass phpunit.xml configuration values to filter and code coverage classes.

Merge branch 'phpunit9coverage' of https://github.com/BlairCooper/phing…

eaf58de

… into phpunit9coverage

BlairCooper added 3 commits November 12, 2020 19:56

Additional unit tests for PHPUnitTask

985e2da

Additional unit tests for PHPUnitTask

cfb633d

Merge branch 'phpunit9coverage'

e2e1108

siad007 self-requested a review November 14, 2020 11:23

siad007 assigned siad007 and BlairCooper Nov 14, 2020

siad007 added enhancement ext tasks labels Nov 14, 2020

siad007 added this to the 3.0.0-rc1 milestone Nov 14, 2020

siad007 reviewed Nov 14, 2020

View reviewed changes

siad007 added the needs documentation label Nov 14, 2020

siad007 reviewed Nov 14, 2020

View reviewed changes

siad007 requested changes Nov 14, 2020

View reviewed changes

siad007 reviewed Nov 14, 2020

View reviewed changes

Cleanup related to pull request review

8edaa1d

siad007 mentioned this pull request Nov 14, 2020

Added coverage-* related test setup #1432

Merged

Merge branch 'master' into phpunit9coverage

29eba89

BlairCooper closed this Nov 14, 2020

BlairCooper deleted the phpunit9coverage branch November 14, 2020 20:54

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Make use of PHPUnit 9 code coverage and reporting #1431

Make use of PHPUnit 9 code coverage and reporting #1431

BlairCooper commented Nov 13, 2020

codecov bot commented Nov 13, 2020 •

edited

Loading

siad007 Nov 14, 2020

BlairCooper Nov 14, 2020

siad007 Nov 14, 2020

BlairCooper Nov 14, 2020

siad007 Nov 14, 2020

BlairCooper Nov 14, 2020

siad007 Nov 14, 2020

BlairCooper Nov 14, 2020

siad007 Nov 14, 2020

BlairCooper Nov 14, 2020

siad007 Nov 14, 2020

BlairCooper Nov 14, 2020

siad007 left a comment •

edited

Loading

siad007 Nov 14, 2020 •

edited

Loading

BlairCooper Nov 14, 2020

siad007 Nov 14, 2020

BlairCooper Nov 14, 2020

siad007 Nov 14, 2020

siad007 Nov 14, 2020

siad007 Nov 14, 2020

BlairCooper Nov 14, 2020

siad007 Nov 14, 2020

siad007 commented Nov 14, 2020

BlairCooper commented Nov 14, 2020

		@@ -1,4 +1,6 @@
		<?php
		use PHPUnit\TextUI\XmlConfiguration\CodeCoverage\CodeCoverage;

Make use of PHPUnit 9 code coverage and reporting #1431

Make use of PHPUnit 9 code coverage and reporting #1431

Conversation

BlairCooper commented Nov 13, 2020

codecov bot commented Nov 13, 2020 • edited Loading

Codecov Report

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

siad007 left a comment • edited Loading

Choose a reason for hiding this comment

siad007 Nov 14, 2020 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

siad007 commented Nov 14, 2020

BlairCooper commented Nov 14, 2020

codecov bot commented Nov 13, 2020 •

edited

Loading

siad007 left a comment •

edited

Loading

siad007 Nov 14, 2020 •

edited

Loading