|
| 1 | +# Reporting bugs |
| 2 | + |
| 3 | +## Writing an effective bug report |
| 4 | + |
| 5 | +When reporting a bug, start with the question |
| 6 | +`What does a bug report need to tell the developer`. |
| 7 | + |
| 8 | +Generally you want the following parts covered: |
| 9 | + |
| 10 | +- What is the problem |
| 11 | + |
| 12 | +- How can the developer reproduce the problem (to see it for themselves), to |
| 13 | + bisect when it was introduced or to find if it got fixed already. |
| 14 | + |
| 15 | +- At what point does the problem occur |
| 16 | + |
| 17 | +- What environment did this occur in |
| 18 | + |
| 19 | +Make sure the above items are covered and the bug is easy to review and parse: |
| 20 | + |
| 21 | +- **Title** should clearly describe the problem. Bugs are often sorted from |
| 22 | + the issue list which only contains the title |
| 23 | + |
| 24 | +- **Logs** should generally be attachments (drag & drop or click on bottom bar |
| 25 | + when entering issue text) and not inline with the issue. |
| 26 | + |
| 27 | +- **Reproduction steps** and **environment** should be clearly highlighted. If |
| 28 | + running commands reproduce the issue (very common), the commands should be |
| 29 | + in a code block/script format. |
| 30 | + |
| 31 | +### Describing the problem |
| 32 | + |
| 33 | +Make sure the issue is obvious or provide a link explaining why the expected |
| 34 | +result is not met. |
| 35 | + |
| 36 | +Examples: |
| 37 | + |
| 38 | +- `(Core dump) seen` is obvious since there should be no core dumps |
| 39 | + |
| 40 | +- `Failure trying to read attribute X in cluster Y which is marked MANDATORY in the spec` |
| 41 | + references the spec and describes why attribute read should succeed. |
| 42 | + |
| 43 | +- `Failure trying to write attribute X in cluster Y, which is enabled since cluster FeatureMap enabled X and spec describes as writable.` |
| 44 | + references the spec and explicitly states that an optional attribute is |
| 45 | + enabled based on device status |
| 46 | + |
| 47 | +- `Running certification test TC-A-B-C (link included) fails at step 3: test case asks for command to succeed, I get ACCESS_DENIED instead` |
| 48 | + describes a pre-defined test case that is expected to pass but fails. Note |
| 49 | + that full link to the test description is needed (and should be covered by |
| 50 | + 'how to reproduce' part) |
| 51 | + |
| 52 | +Unless manually curated (e.g. few lines showing the problem), logs should be |
| 53 | +always attachments and not inlined in the bug as the make the bug report too |
| 54 | +long. |
| 55 | + |
| 56 | +### Reproduction steps and when does the issue occur |
| 57 | + |
| 58 | +Include all steps needed to reproduce the problem. Link any supporting |
| 59 | +documentation. |
| 60 | + |
| 61 | +If stating something of the form `TC-A-B-C step 4 fails` then there should be a |
| 62 | +link to TC-A-B-C and ideally a list of the commands of each step since test |
| 63 | +cases may change over time. |
| 64 | + |
| 65 | +The bug report should contain all the information for a developer to reproduce |
| 66 | +the issue without needing to have access to CHIP Test case repository (not |
| 67 | +everyone does) |
| 68 | + |
| 69 | +### Environment for reproduction |
| 70 | + |
| 71 | +Assume that the developer will want to reproduce the issue and will try to |
| 72 | +mirror your environment to a reasonable degree. For this, at a minimum the |
| 73 | +platforms on which everything is running is needed. |
| 74 | + |
| 75 | +Try to provide as much information as seems relevant. At a minimum this could |
| 76 | +look like |
| 77 | +`Failed to commission nrf board using chip-tool running on linux. Used build on SHA abcde...`. |
| 78 | +This provides basic information (use nrf board, use chip-tool on linux, default |
| 79 | +build) to get started. Beyond that, you can refine if more items seem relevant: |
| 80 | + |
| 81 | +- `Tested on TE9` or `Tested on interop branch xyz` gives a build reference |
| 82 | + point. Useful when branches/tags are used instead of master branch as |
| 83 | + development happens on master branch. |
| 84 | + |
| 85 | +- `Thread devices fail, tested with qpg and efr32` shows that this seems to be |
| 86 | + a general thread issue and developer can investigate on multiple of them |
| 87 | + |
| 88 | +* `Tested with avahi-build and it passes/fails` helps the developer with |
| 89 | + information of non-default builds that pass/fail to narrow down the problem |
| 90 | + |
| 91 | +* `Passes with chip-tool-darwin and repl but fails with chip-tool` helps the |
| 92 | + developer in narrowing down the problem |
| 93 | + |
| 94 | +### Additional information |
| 95 | + |
| 96 | +Providing additional information that can be helpful is encouraged. Each bug |
| 97 | +report is different here. Some examples: |
| 98 | + |
| 99 | +- `This worked last week (around Jan 5) but started failing in recent master builds` |
| 100 | + |
| 101 | +- `Specification changed this attribute from optional to mandatory so this may be the cause of the issue` |
| 102 | + |
| 103 | +- `This issue may be related to #1234 as the same error is seen, however the reproduction steps seemed distinct enough that I openend a new issue` |
| 104 | + |
| 105 | +- `While running this, I observed 100% CPU before the operation finally timed out` |
| 106 | + |
| 107 | +- `While running this test, I observed device under test rebooting, logs attached.` |
| 108 | + |
| 109 | +- `This only happens intermitently - I see it about 30% of the time` |
0 commit comments