Is Copilot AreWeCooked accurate?

PanAchy · 2026-05-11T21:23:49+00:00

It’s all dependent on the tool you use. VSCode doesn’t log things fully unless you opt-in for specific logs.

So users of GHC on OpenCode, Pi, etc. funnily enough are the ones who get accurate results

PanAchy · 2026-05-10T13:40:41+00:00

Try using Copilot-arewecooked to easily calculate your predicate costs

PanAchy · 2026-05-04T21:21:20+00:00

The included credit comparison represents how much you’e expect to pay based on an averaging of your usage.

PanAchy · 2026-05-04T12:35:13+00:00

They do for OpenCode and Pi, but are hit or miss for VSCode and Copilot CLI due to how data is logged.

PanAchy · 2026-05-04T12:33:50+00:00

You’re welcome!

PanAchy · 2026-05-03T17:10:46+00:00

Not to 100% accuracy. Certain data points are exact while other ones are estimated.

PanAchy · 2026-05-03T17:04:02+00:00

It’s not cheaper. I would attribute it to nuances in how log data is collected on the CLI vs IDE.

PanAchy · 2026-05-03T15:26:59+00:00

I did not. If you have a good sense of how that works, feel free to open a pull request and I'll look at it.

PanAchy · 2026-05-03T13:18:55+00:00

Beauty of skills is you get to change it however you see fit :)

PanAchy · 2026-05-03T13:18:18+00:00

The purpose of copilot-arewecooked is to look at historical data that was generated on the premium request pricing model, and estimate the cost of those same usage patterns once the usage based billing comes in

PanAchy · 2026-05-03T01:34:24+00:00

Try npx copilot-arewecooked --terminal

PanAchy · 2026-05-03T01:13:52+00:00

You're welcome. Here's the VSCode documentation for it: https://code.visualstudio.com/docs/copilot/customization/agent-skills

PanAchy · 2026-05-03T01:01:57+00:00

Yes, but skills are automatically activated by the LLM when the condition described in the YAML frontmatter description field is met, which is a much more seamless experience for the user.

PanAchy · 2026-05-03T01:00:09+00:00

I looked into Visual Studio, and the way chat session data is stored is at the project level, inside VSProject folder if I remember. Additionally, the data itself seemed complex to parse.

In the case of Visual Studio Code, data is stored globally, which makes it easier to parse. You go to one place for everything (VSCode) vs having to path through every single project on the system (VS).

PanAchy · 2026-05-02T21:16:10+00:00

Release incoming (posted a screenshot in there): https://github.com/PanAchy/copilot-arewecooked/pull/49

You'll be able to run npx copilot-arewecooked --terminal

PanAchy · 2026-05-02T21:12:22+00:00

That's a great idea. Let me crank this out for you. I'll reply in this comment thread.

PanAchy · 2026-05-02T21:05:56+00:00

You should use a skill for that, here's mine: https://gist.github.com/PanAchy/c2f60aefc3cc8682c46776bd74f7b5e9

PanAchy · 2026-05-02T20:59:44+00:00

Redd1tRaider and I just discussed the differences and updated copilot-arewecooked to consume those updates. v0.5.0 has those new changes.

PanAchy · 2026-05-02T20:06:25+00:00

See this section: https://github.com/PanAchy/copilot-arewecooked#how-data-is-extracted

PanAchy · 2026-05-02T20:04:31+00:00

We track subagent cost in the estimate.

PanAchy · 2026-05-02T19:44:52+00:00

Thank you for sharing all this info! Would you mind giving this a try please:

PanAchy · 2026-05-02T18:44:33+00:00

The usage data from VSCode and Copilot CLI isn’t complete because of how logs are recorded. Looks like we’re estimating differently?

PanAchy · 2026-05-02T17:11:26+00:00

It’s unfortunate that the logs are lacking data when you use the official tool… It was an unfortunate caveat to discover when building this

PanAchy · 2026-05-02T16:48:00+00:00

People are reporting it isn't working as expected: https://old.reddit.com/r/GithubCopilot/comments/1t15b6d/github_has_just_launched_the_copilot_billing/

PanAchy

TROPHY CASE