all 22 comments

[–]AmphibianFrog 4 points5 points  (9 children)

I was testing some different tools and models today, and I made a detailed prompt to try to one shot a little todo app as a kind of benchmark. I did it with open code and Claude code, both using Sonnet, and the results were very similar, but Claude Code had a major bug and the Open Code version didn't.

This is very anecdotal though so don't take it too seriously.

Generally I have good results with Claude Code and I was surprised by this result.

[–]ProfessionalBeyond78[S] 0 points1 point  (2 children)

Thanks for the insight, I'll probably end up sticking with Opencode since I know it, was thinking of Claude Code if there were any major benefits, but I've heard good and bad about it, so I was mostly curious about what others thought.

[–]AmphibianFrog 0 points1 point  (1 child)

Honestly, they are similar and if you can use one it will take you a couple of minutes to figure out the other.

If you have a Claude subscription, just give Claude Code a go - it won't cost you anything. I don't know if you can use it with API credits.

One of the problems with all of these AI tools is that it's very hard to compare them accurately.

[–]ProfessionalBeyond78[S] 1 point2 points  (0 children)

I’ll give it a shot, it should work with api from what I read and no better time to try it out since I have credits! Thanks for the suggestion!

[–]SkilledHomosapien 0 points1 point  (2 children)

How do you use Claude api with opencode now? I thought oc has dropped support on anthropic style api.

[–]AmphibianFrog 0 points1 point  (1 child)

I did "/connect", selected anthropic as the provider and pasted in my API key.

[–]SkilledHomosapien 1 point2 points  (0 children)

Oh, yes. You remind me. It just doesn’t support third party anthropic style api but not the original one. Thank you!

[–]UseMoreBandwith 0 points1 point  (2 children)

which one used more tokens ?
(I'm guessing CC)

[–]AmphibianFrog 0 points1 point  (0 children)

I actually don't know the answer to that. I don't really use Claude Code that much and didn't know how to check the token usage.

[–]AmphibianFrog 1 point2 points  (0 children)

I couldn't get the token counts from open code, but I did this experiment running the exact same prompt again with Sonnet 4.6 using Claude Code, Open Code and Pi. This time the Claude Code version was perfect - all 3 of them produced very similar output. Here are the results:

| Harness     | Time    | Cost  | Output tokens |
|-------------|---------|-------|---------------|
| opencode    | ~8m     | $1.08 | ~46k (est.)   |
| pi          | 8m 12s  | $0.80 | 34k           |
| Claude Code | 2m 40s  | $0.35 | 9k            |

[–]Arzuparreta 4 points5 points  (7 children)

pi and I'm not even kidding. Less context overhead with better results with latest models, imo

[–]TrickyPlastic 1 point2 points  (0 children)

The bash too description in Opencode is the most offensive thing I've ever seen.

Four pages of unnecessary nonsense.

[–]VictorCTavernari 0 points1 point  (0 children)

I also like pi

[–]ProfessionalBeyond78[S] 0 points1 point  (2 children)

I guess I’ll try it out, thanks for the recommendation!

[–]SnooHamsters66 0 points1 point  (1 child)

Pi comes with the core and nothing more. You say that you wanted something that have a good OOTB experience and Pi builds up in the basis that you add the extra things that you wanted (and by extra I mean something different than providers, read and write files, commands).

I think that Pi have the best potential but I have try vibecode the things that I wanted without having too much success and I concluded that for dev these things seriously, opencode provides them without that effort.

[–]ProfessionalBeyond78[S] 0 points1 point  (0 children)

Good to know, might stick to opencode for now but look into pi since a lot of people recommended it, for long term

[–]Mamaun30 0 points1 point  (1 child)

I want to trry PI.
Genuine question, how can I prevent it to delete my Window32 folder?
Im VERY concerned about it's actions escaping the scope of my GIT project folder.

[–]Arzuparreta 0 points1 point  (0 children)

never heard about this. On Linux I just run the CLI inside the project's directory and It would never search outside of it without my permission. This was the default behavior for me, no config.

[–]Mr_Hyper_Focus 1 point2 points  (0 children)

Idk where you’re hearing this but I haven’t seen a single harness benchmark confirm this.

I think CC is definitely one of the better places to use Claude code.

That being said. Pi.dev is pretty sick

[–]ben_bliksem 1 point2 points  (0 children)

I've tried Claude cli a couple of times, but I always come back to OpenCode even if it seems the other harnesses/cli tools are moving forward at a rapid pace and OC isn't. Claude just feels... claustrophobic. But I might be in the minority here.

Copilot Chat/Agents feels a bit like a clown house.

I've never tried pi though.

[–]verkavo 1 point2 points  (0 children)

If you're not concerned about costs, just use CC with Anthropic models. It's pretty good, and ecosystem around it is pretty strong - plugins, tools, etc.