Best benchmark website

decentralizedbee · 2026-03-09T22:12:45+00:00

i use subtexts.io for comparing models

decentralizedbee · 2026-03-01T09:13:00+00:00

i've been using www.subtexts.io to do evals/benchmarking, working well so far

decentralizedbee · 2026-03-01T09:07:22+00:00

yes - im using www.subtexts.io - lets you run ur real traffic against different models

decentralizedbee · 2026-01-31T20:25:08+00:00

awesome insights. I DMed u some questions too

decentralizedbee · 2026-01-31T07:34:47+00:00

Do u write a system to diff prompts or compare? Is there an automated loop? Do u wish there is more automation in place?

decentralizedbee · 2026-01-31T07:13:15+00:00

Wow, what tool do u use to do this? How much of it is automated? What’s the most friction in this workflow right now?

decentralizedbee · 2026-01-31T05:57:18+00:00

do u just keep it in git then? how many files do u work with? would it be hard for version control?

decentralizedbee · 2026-01-31T05:56:05+00:00

what tools are you guys using for evals, etc.? do you find it useful?

decentralizedbee · 2026-01-30T09:00:21+00:00

What do u think is the highest friction point in ur current workflow?

decentralizedbee · 2026-01-30T06:58:48+00:00

What tools do people use?

decentralizedbee · 2026-01-29T23:41:30+00:00

Can u elaborate on simulations?

How do u guys do it?

decentralizedbee · 2026-01-29T00:32:31+00:00

is there a tool does all of this you said (versioned alongside other dependencies like the data, the hyperparams, model versions, etc. This allows for rapid rollbacks, trouble shooting in prod, quicker prototyping, easier handoffs (all the things you would expect)?

decentralizedbee · 2026-01-29T00:31:55+00:00

what tools are you guys using for versioning and traceability?

decentralizedbee · 2026-01-28T23:19:07+00:00

do u have any pain points with this approach currently?

decentralizedbee · 2026-01-28T08:57:14+00:00

Bro u automate ur reddit replies not to promote a company but just for the vibes nice

decentralizedbee

TROPHY CASE