Skip to content

Commit

Permalink
Update rmw_fastrtps_cpp and rmw_fastrtps_shared_cpp QDs to QL2. (#456)
Browse files Browse the repository at this point in the history
Signed-off-by: Michel Hidalgo <michel@ekumenlabs.com>
  • Loading branch information
hidmic authored Sep 30, 2020
1 parent 23f9c4c commit 7bfe150
Show file tree
Hide file tree
Showing 2 changed files with 29 additions and 11 deletions.
20 changes: 15 additions & 5 deletions rmw_fastrtps_cpp/QUALITY_DECLARATION.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,9 +2,9 @@ This document is a declaration of software quality for the `rmw_fastrtps_cpp` pa

# `rmw_fastrtps_cpp` Quality Declaration

The package `rmw_fastrtps_cpp` claims to be in the **Quality Level 3** category.
The package `rmw_fastrtps_cpp` claims to be in the **Quality Level 2** category.

Below are the rationales, notes, and caveats for this claim, organized by each requirement listed in the [Package Requirements for Quality Level 3 in REP-2004](https://www.ros.org/reps/rep-2004.html).
Below are the rationales, notes, and caveats for this claim, organized by each requirement listed in the [Package Requirements for Quality Level 2 in REP-2004](https://www.ros.org/reps/rep-2004.html).

## Version Policy [1]

Expand Down Expand Up @@ -86,11 +86,18 @@ The results of the test can be found [here](https://ci.ros2.org/view/nightly/job

### Feature Testing [4.i]

All `rmw_fastrtps_cpp` public features are ROS middleware features. Integration and system tests up the stack, such as those found in [`test_rclcpp`](https://github.com/ros2/system_tests/tree/master/test_rclcpp) and [`test_communication`](https://github.com/ros2/system_tests/tree/master/test_communication) packages, provide coverage. Nightly CI jobs in [`ci.ros2.org`](https://ci.ros2.org/) and [`build.ros2.org`](https://build.ros2.org/), where `rmw_fastrtps_cpp` is the default `rmw` implementation, thoroughly exercise this middleware.
All `rmw_fastrtps_cpp` public features are ROS middleware features.

Unit, integration, and system tests higher up in the stack, such as those found in [`test_rmw_implementation`](https://github.com/ros2/rmw_implementation/tree/master/test_rmw_implementation), [`test_rclcpp`](https://github.com/ros2/system_tests/tree/master/test_rclcpp), and [`test_communication`](https://github.com/ros2/system_tests/tree/master/test_communication) packages, provide feature coverage. Nightly CI jobs in [`ci.ros2.org`](https://ci.ros2.org/) and [`build.ros2.org`](https://build.ros2.org/), where `rmw_fastrtps_cpp` is the default `rmw` implementation, thoroughly exercise this middleware.

### Public API Testing [4.ii]

There is no API testing in `rmw_fastrtps_cpp`.
`rmw_fastrtps_cpp` implements the ROS middleware public API, but also provides public API of its own.

Unit tests located in the [`test`](https://github.com/ros2/rmw_fastrtps/tree/master/rmw_fastrtps_cpp/test) directory provide coverage for public but `rmw_fastrtps_cpp` specific API.
New additions or changes to this API require tests before being added.

Unit, integration, and system tests higher up in the stack, such as those found in [`test_rmw_implementation`](https://github.com/ros2/rmw_implementation/tree/master/test_rmw_implementation), [`test_rclcpp`](https://github.com/ros2/system_tests/tree/master/test_rclcpp), and [`test_communication`](https://github.com/ros2/system_tests/tree/master/test_communication) packages, ensure compliance with the ROS middleware API specification (see [`rmw`](https://github.com/ros2/rmw) package) and further extend coverage.

### Coverage [4.iii]

Expand All @@ -104,7 +111,10 @@ This includes:

Changes are required to make a best effort to keep or increase coverage before being accepted, but decreases are allowed if properly justified and accepted by reviewers.

Current coverage statistics can be viewed [here](https://ci.ros2.org/job/ci_linux_coverage/lastSuccessfulBuild/cobertura/src_ros2_rmw_fastrtps_rmw_fastrtps_cpp_src/). This package does not yet meet the 95% coverage guideline.
Current coverage statistics can be viewed [here](https://ci.ros2.org/job/ci_linux_coverage/lastSuccessfulBuild/cobertura/src_ros2_rmw_fastrtps_rmw_fastrtps_cpp_src/).

This package claims to meet the coverage requirements for the current quality level, even though it doesn't have 95% line coverage.
The justification is that the only uncovered lines have to do with system resource exhaustion and Fast-DDS internal failure.

A summary of how these statistics are calculated can be found in the [ROS 2 On-boarding guide](https://index.ros.org/doc/ros2/Contributing/ROS-2-On-boarding-Guide/#note-on-coverage-runs).

Expand Down
20 changes: 14 additions & 6 deletions rmw_fastrtps_shared_cpp/QUALITY_DECLARATION.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,9 +2,9 @@ This document is a declaration of software quality for the `rmw_fastrtps_shared_

# `rmw_fastrtps_shared_cpp` Quality Declaration

The package `rmw_fastrtps_shared_cpp` claims to be in the **Quality Level 3** category.
The package `rmw_fastrtps_shared_cpp` claims to be in the **Quality Level 2** category.

Below are the rationales, notes, and caveats for this claim, organized by each requirement listed in the [Package Requirements for Quality Level 3 in REP-2004](https://www.ros.org/reps/rep-2004.html).
Below are the rationales, notes, and caveats for this claim, organized by each requirement listed in the [Package Requirements for Quality Level 2 in REP-2004](https://www.ros.org/reps/rep-2004.html).

## Version Policy [1]

Expand Down Expand Up @@ -86,12 +86,18 @@ The results of the test can be found [here](https://ci.ros2.org/view/nightly/job

### Feature Testing [4.i]

Many of the features have testing in `rmw_fastrtps_shared_cpp`. Since `rmw_fastrtps_shared_cpp` supports `rmw_fastrtps_cpp` and `rmw_fastrtps_dynamic_cpp`, integration and system tests for ROS middlewares up the stack, such as those found in [`test_rclcpp`](https://github.com/ros2/system_tests/tree/master/test_rclcpp) and [`test_communication`](https://github.com/ros2/system_tests/tree/master/test_communication) packages, provide coverage.
All `rmw_fastrtps_shared_cpp` public features support ROS middleware feature implementations in `rmw_fastrtps_cpp` and `rmw_fastrtps_dynamic_cpp`.

Unit, integration, and system tests for ROS middlewares higher up in the stack, such as those found in [`test_rmw_implementation`](https://github.com/ros2/rmw_implementation/tree/master/test_rmw_implementation), [`test_rclcpp`](https://github.com/ros2/system_tests/tree/master/test_rclcpp), and [`test_communication`](https://github.com/ros2/system_tests/tree/master/test_communication) packages, provide coverage.

### Public API Testing [4.ii]

Many parts of the public API have testing in `rmw_fastrtps_shared_cpp`.
The latest nightly test results can be seen [here](https://ci.ros2.org/view/nightly/job/nightly_linux_release/lastBuild/testReport/rmw_fastrtps_shared_cpp/).
All `rmw_fastrtps_shared_cpp` public API supports API implementations in in `rmw_fastrtps_cpp` and `rmw_fastrtps_dynamic_cpp`.

Unit tests for many parts of this API provide coverage, and they are located in the [`test`](https://github.com/ros2/rmw_fastrtps/tree/master/rmw_fastrtps_shared_cpp/test) directory.
New additions or changes to this API require tests before being added.

Unit, integration, and system tests for `rmw_fastrtps_cpp` and `rmw_fastrtps_dynamic_cpp` higher up in the stack, such as those found in [`test_rmw_implementation`](https://github.com/ros2/rmw_implementation/tree/master/test_rmw_implementation), [`test_rclcpp`](https://github.com/ros2/system_tests/tree/master/test_rclcpp), and [`test_communication`](https://github.com/ros2/system_tests/tree/master/test_communication) packages, further extend coverage.

### Coverage [4.iii]

Expand All @@ -110,7 +116,9 @@ Current coverage statistics can be viewed
[here](https://ci.ros2.org/job/ci_linux_coverage/lastSuccessfulBuild/cobertura/src_ros2_rmw_fastrtps_rmw_fastrtps_shared_cpp_include_rmw_fastrtps_shared_cpp/),
[here](https://ci.ros2.org/job/ci_linux_coverage/lastSuccessfulBuild/cobertura/src_ros2_rmw_fastrtps_rmw_fastrtps_shared_cpp_src/), and
[here](https://ci.ros2.org/job/ci_linux_coverage/lastSuccessfulBuild/cobertura/src_ros2_rmw_fastrtps_rmw_fastrtps_shared_cpp_src_types/).
This package does not yet meet the 95% coverage guideline.

This package claims to meet the coverage requirements for the current quality level, even though it doesn't have 95% line coverage.
The justification is that the only uncovered lines have to do with system resource exhaustion and Fast-DDS internal failure.

A summary of how these statistics are calculated can be found in the [ROS 2 On-boarding guide](https://index.ros.org/doc/ros2/Contributing/ROS-2-On-boarding-Guide/#note-on-coverage-runs).

Expand Down

0 comments on commit 7bfe150

Please sign in to comment.