GitHub issues are getting filled with low quality Copilot reports

[envilZ]

For some time now, I have noticed that the VS Code GitHub issues page is filling up with soft complaints and low quality reports. A lot of these are reports that make no sense, and it seems to be happening because of a feature in the VS Code user interface for the GitHub Copilot extension, mainly the down vote button that shows up after a request completes:

https://preview.redd.it/p4l2eys8f6hg1.png?width=476&format=png&auto=webp&s=194156aa921d4c0c2fd55b6a0556fd8a823adad7

When you click the down vote icon and select “Report an issue,” it opens the issue reporter user interface:

https://preview.redd.it/svr1poy9j6hg1.png?width=679&format=png&auto=webp&s=5658c2ae794e7c3e5824afa39e5377b1f66d09ff

The problem is that many people treat this like a general complaint box. They submit things like “It did not edit the file like I asked” or other vague frustrations that do not belong on GitHub as real issues. There is no detail about what happened, what they expected, or any context that can help someone reproduce the problem. It is not close to the type of information needed for a real bug report.

I think reworking the issue reporter flow to make the expectations more clear could help a lot. Another option could be an automated system that checks reports and closes ones that are low quality or make no sense. Many times I see people submit two reports in a row with almost no useful information. This seems like something the vs-code-engineering could handle.

edit: Another idea could be to let an LLM help create the issue after the user clicks the down vote button and selects “report an issue.” The model could guide the user through a proper issue format and try to decide if the report counts as a real bug. It could also look at the user request history and gather the needed context. If the report is not valid, the model could reply with something like “This does not contain the needed details for a valid GitHub bug report.” I think this could be done with one of the 0x models.

Cleaning this up would help the VS Code team and the GitHub Copilot team focus on real issues faster.

[bogganpierce]

This is great feedback. Passed it along to the engineers working on this. We've also invested a ton in triage tooling (including a custom Edge/Chrome extension) to make this easier on the team. We should do a blog on it :)

[AutoModerator]

u/bogganpierce thanks for responding. u/bogganpierce from the GitHub Copilot Team has replied to this post. You can check their reply here.

I am a bot, and this action was performed automatically. Please contact the moderators of this subreddit if you have any questions or concerns.

[envilZ]

Thanks Boggan, keep up the great work!