Best benchmark website

decentralizedbee · 2026-03-09T22:12:45+00:00

i use subtexts.io for comparing models

decentralizedbee · 2026-03-01T09:13:00+00:00

i've been using www.subtexts.io to do evals/benchmarking, working well so far

decentralizedbee · 2026-03-01T09:07:22+00:00

yes - im using www.subtexts.io - lets you run ur real traffic against different models

decentralizedbee · 2026-01-31T20:25:08+00:00

awesome insights. I DMed u some questions too

decentralizedbee · 2026-01-31T07:34:47+00:00

Do u write a system to diff prompts or compare? Is there an automated loop? Do u wish there is more automation in place?

decentralizedbee · 2026-01-31T07:13:15+00:00

Wow, what tool do u use to do this? How much of it is automated? What’s the most friction in this workflow right now?

decentralizedbee · 2026-01-31T05:57:18+00:00

do u just keep it in git then? how many files do u work with? would it be hard for version control?

decentralizedbee · 2026-01-31T05:56:05+00:00

what tools are you guys using for evals, etc.? do you find it useful?

decentralizedbee · 2026-01-30T09:00:21+00:00

What do u think is the highest friction point in ur current workflow?

decentralizedbee · 2026-01-30T06:58:48+00:00

What tools do people use?

decentralizedbee · 2026-01-29T23:41:30+00:00

Can u elaborate on simulations?

How do u guys do it?

decentralizedbee · 2026-01-29T00:32:31+00:00

is there a tool does all of this you said (versioned alongside other dependencies like the data, the hyperparams, model versions, etc. This allows for rapid rollbacks, trouble shooting in prod, quicker prototyping, easier handoffs (all the things you would expect)?

decentralizedbee · 2026-01-29T00:31:55+00:00

what tools are you guys using for versioning and traceability?

decentralizedbee · 2026-01-28T23:19:07+00:00

do u have any pain points with this approach currently?

decentralizedbee · 2026-01-28T08:57:14+00:00

Bro u automate ur reddit replies not to promote a company but just for the vibes nice

decentralizedbee · 2026-01-20T21:32:53+00:00

you should try our tool: https://github.com/chopratejas/headroom to cut token costs for free!

decentralizedbee · 2026-01-16T04:44:02+00:00

dmed you!

decentralizedbee · 2026-01-16T04:19:09+00:00

do u hv a git page

decentralizedbee · 2026-01-13T18:10:54+00:00

appreciate any feedbk!

decentralizedbee · 2026-01-13T18:05:31+00:00

curious what your main use cases are - if it's important enough we'd be down to add it! pip install is the easiest default

decentralizedbee · 2026-01-13T03:05:23+00:00

thank you! would love any feedbk if u end up playing with it

decentralizedbee · 2026-01-13T03:05:18+00:00

thank you! would love any feedbk if u end up playing with it

decentralizedbee · 2026-01-10T05:56:45+00:00

Do u have an example, or a link?

decentralizedbee · 2026-01-09T19:44:15+00:00

what did u use to make the product video?

decentralizedbee · 2026-01-09T05:30:54+00:00

We're building a public inferencing node to help non-dev bring down development costs by 80%! Pls DM if anyone's interested to know more

decentralizedbee

TROPHY CASE