Announcement: Extending API review process #74273
Comments
|
Tagging subscribers to this area: @dotnet/area-meta Issue DetailsWe regular review API requests, but our usual process focusses on two stages: triage and review meetings. We don't really keep track of issues that need work or requests that were approved and now need to be done. Here is the full funnel with counts and links to Issues of .NET: graph TD
S(<a href='https://issuesof.net/?q=is%3Aopen%20is%3Aissue%20label%3Aapi-suggestion'>Suggested</a><br>924) --> T{Triage}
T -->R(<a href='https://issuesof.net/?q=is%3Aopen%20is%3Aissue%20label%3Aapi-ready-for-review'>Ready for Review</a><br>41)
T -->C(<a href='https://issuesof.net/?q=is%3Aclosed%20is%3Aissue%20%28label%3Aapi-ready-for-review%20or%20label%3Aapi-needs-work%20or%20label%3Aapi-suggestion%29'>Closed</a><br>1,936)
R --> RM{Review Meeting}
RM --> A(<a href='https://issuesof.net/?q=is%3Aopen%20is%3Aissue%20label%3Aapi-approved'>Approved</a><br>117)
RM --> N(<a href='https://issuesof.net/?q=is%3Aopen%20is%3Aissue%20label%3Aapi-needs-work'>Needs Work</a><br>258)
RM --> C
N --> T
We're proposing a process to drive down the number of suggested, approved, and needs work items. Needs WorkLet's start with issues we reviewed and decided they need work. 75% of those issues were updated more than three months ago, which I would consider as stale. But we can also start with issues older than 6 months, which is still 65%.
Our suggestion is we apply the label ApprovedThis bucket is a bit harder. Some of these are quite old, but most of them were approved within the last year. If we look for a cut off line it looks like this:
We shouldn't close those because we spent time on them and decided that they have merit. So we suggest that we triage them and decide which bucket they are in:
SuggestedThat's the biggest bucket. 67% of issues are older than three months.
We suggest we apply the same process as for Needs Work. The remaining bucket we should ensure has an area label such that it can be triaged by area pod owners. Currently there are 34 issues without area. Any thoughts on this?
|
terrajobst
removed
the
untriaged
|
When the 15 days timer is started will the author be tagged automatically, so he is informed? Otherwise it may be closed without anyone noticing |
I don't think our bot has the capability to do this right now. It assumes the author is subscribed to their own issue, which I assume is the default. Example #73303 (comment) |
|
BTW, it's 14 days, not that it matters terribly. |
I made an API proposal which first got approved and then got put back, because a team member wasn't happy about the implementation. This was on April this year and now its just "without activity". It does not fall under these six months but soon it will. I'm not quite sure how a 15 days timer will help, since it's not me everyone's waiting for. Rather than this team member to explain how exactly the implementation should be instead. Maybe instead of just blindly attach this label to all older issues first check whats the status? Otherwise you will attach it, author (like me) gets notified. We reply, label gets removed but issue continue to stay there until a team member finally checks the status himself. So in other words: nothing changes, except issue got a forced unnecessary offtopic comment from author to remove the label. |
|
Some suggestions are not responded by area owners. This may be heavy work for them, but we should at least give a first-impression triage before marking as no activity. |
You are right @huoyaoyuan . I do only see 5 labeled untriaged - but I am guessing you have in mind the proposals that we looked at but the engagement was not very deep. The scale is large, so focus will help us here. API suggestions tend to be driven by their authors, and typically we would not take them further if the author was no longer interested. So if an automated process can help close that sub-set, it will identify the ones worth more attention? |
|
Let's take my issue #53354 as an example. It was responded as "not bad", but no further action was taken to bring it ready for review. I can't see there's anything to do as the author. |
|
Right, in that case the comments suggest we missed to mark it ready. |
@Symbai You mention a common scenario, and one that we know is frustrating as an external contributor. A lot of times when these situations arise, we've found it was unclear who was expected to take the next action. Optimistically, forcing someone to comment to keep the issue open will yield a comment (from the author, a community member, or a team member) citing what's needed next. @terrajobst I edited the issue description to normalize on days instead of months, and to correct it from 15 days to 14 days per the comment above. |
|
@jeffhandley I was talking about this one #60903 (comment) where stephentoub was not happy with the PR. Another member of the .NET team closed the PR but that's it. For me its not clear what stephentoub was missing. Besides of that, it is not clear to me if this needs a new review process or if it needs any action from my side etc. After that, there has been 6 comments already but none made by stephentoub or anyone else who felt unhappy with the PR. Thats why I am saying that adding this auto label process blindly on all older proposals is not a good idea. It should be done manually by checking if it really needs the attention of the author. Anyway I sent a ping comment... |
I thought my feedback in the discussion at #62375 (comment) was fairly clear, and effectively it was that the API as approved was flawed and it needs to be revisited. When @jozkee closed the PR, he also removed the api-approved from the issue and replaced it with api-needs-work, highlighting that the next step is revisiting the API to determine if/how it should be fixed. The next step is that someone who really cares about this API needs to revisit it to push it forward. |
I care. I revisted. I commented to push it forward. Still I see no activity from the .NET team about this at all. Your feedback was not clear, sorry to say. It still isn't. WHO we are waiting for? Someone from the .NET team? And what should this person do then? I thought that it has been brought back so the persons (including you) who are unhappy about the reviewed implementation are now discussing how it should be implemented instead but that obviously isn't going to happen. Instead it looks like everyone jumped off the proposal and for me, the author and someone who is really interested into getting this implemented, doesn't know what to do. I can add a comment to it all night long but that doesn't seem to be helpful at this time. |
|
I hear your frustration @Symbai about that particular PR. Let's reconnect on that and keep the discussion here about how to best manage the over 1000 api-* issues. With such a large number of issues, there will be miscommunications and glitches on some perhaps. But we have to use automated systems to help us, if we went through all these manually we would not have time to fix bugs, add features, etc etc. |
|
I too have been waiting for a few APIs for a long time that could be useful in PowerShell and beyond and that are in some obscure state. :-( It should be noted that most approved APIs are quickly implemented by the .Net Team. I assume that most of the approved 116 APIs have not been implemented because they are moved to the next milestone. The main stopper for third party developers to implement some new API I guess is cross-platform. Maybe we should think of ways to simplify this. For example, put all the pinvokes into a separate project which can be used not only by .Net Runtime but also by other projects like PowerShell. |
|
Do others have thoughts or feedback? Several comments above pointed to proposals that have not gotten enough attention for some reason - my feeling is that the largest reason is scale. There are so many relative to the number of area owners that we inevitably overlook good proposals or conversation tails off. Consider |
Perhaps "Closed" stage could has specific label (for automatic issue closing and more easy tracking ) |
At the very least allow re-opening of a closed 'stale' issue when someone replies after automatic closure. |
|
@terrajobst is there anything to share about whether you plan to go ahead with this, and what sort of timeline? |
This depends on timeframe since the issue was closed, how long the issue is, whether the top issue requires changes/more work, etc. In many cases, it is simpler and better to open a new issue in the correct format and to cross-link back to the old issue. After having done a lot of triage over old issues in my own areas, the vast majority are "stale" because the original author doesn't follow the API Proposal Template: https://github.com/dotnet/runtime/issues/new?assignees=&labels=api-suggestion&template=02_api_proposal.yml&title=%5BAPI+Proposal%5D%3A+ They will most often just toss up 1-2 paragraphs briefly describing a loose idea without actually going through the effort of providing examples or even describing the surface of the APIs actually needed. |
|
I think this is a good suggestion. Tagging an issue as 'stale' will often prompt the author to tag an MS member and ask what's going on, which is a perfect opportunity for the MS dev to say what actions the author needs to take next 🙂 |
|
Closing as I believe this is complete. |
ghost
locked as resolved and limited conversation to collaborators
We regularly review API requests, but our usual process focuses on two stages: triage and review meetings. We don't really keep track of issues that need work or requests that were approved and now need to be done.
Here is the full funnel with counts and links to Issues of .NET:
graph TD S(<a href='https://issuesof.net/?q=is%3Aopen%20is%3Aissue%20label%3Aapi-suggestion'>Suggested</a><br>924) --> T{Triage} T -->R(<a href='https://issuesof.net/?q=is%3Aopen%20is%3Aissue%20label%3Aapi-ready-for-review'>Ready for Review</a><br>41) T -->C(<a href='https://issuesof.net/?q=is%3Aclosed%20is%3Aissue%20%28label%3Aapi-ready-for-review%20or%20label%3Aapi-needs-work%20or%20label%3Aapi-suggestion%29'>Closed</a><br>1,936) R --> RM{Review Meeting} RM --> A(<a href='https://issuesof.net/?q=is%3Aopen%20is%3Aissue%20label%3Aapi-approved'>Approved</a><br>117) RM --> N(<a href='https://issuesof.net/?q=is%3Aopen%20is%3Aissue%20label%3Aapi-needs-work'>Needs Work</a><br>258) RM --> C N --> TWe're proposing a process to drive down the number of suggested, approved, and needs work items.
Needs Work
Let's start with issues we reviewed and decided they need work. 75% of those issues were updated more than 90 days ago, which I would consider as stale. But we can also start with issues older than 180 days, which is still 65%.
Our suggestion is we apply the label
no-recent-activityorbacklog-cleanup-candidateto all issues older than 180 days. This will start the timer for 14 days waiting for a reaction. After 14 days our automation will post a comment, indicating that it is stale. After 15 more days of inactivity, the issue will be closed. Essentially, this would follow our issue bankruptcy process.Approved
This bucket is a bit harder. Some of these are quite old, but most of them were approved within the last year. If we look for a cut-off line it looks like this:
We shouldn't close those because we spent time on them and decided that they have merit. So we suggest that we triage them and decide which bucket they are in:
help wanted(formerlyup-for-grabs) and move to Future milestone.Suggested
That's the biggest bucket. 67% of issues are older than 90 days.
We suggest we apply the same process as for Needs Work. The remaining bucket we should ensure has an area label such that it can be triaged by area owners. Currently there are 34 issues without area.
Any thoughts on this?
/cc @dotnet/fxdc
The text was updated successfully, but these errors were encountered: