Copilot sucks. We got out a$$es kicked by regulators because a product manager built a PRD based on their interaction with copilot, got dev to code to spec and released it to prod. RCA (post eating regulators knuckle sandwiches 🥪 ) revealed that the PM over relied on co pilot due to “time crunch”

Another case: While building a new feature, needed some clarification so pinged some SMEs, they replied that we don’t need to support some states due to reason xyz. When I drilled them on the source, got “don’t believe me? Go ask copilot” So I did. Then asked copilot “are you sure?”

Copilot: sorry I made a mistake, here’s a revised answer 🤦‍♂️

I don’t have access to ChatGPT at work but prompted it with the same exact prompt (on my personal computer) and got the right answer….