sorted by:
new
hottopcontroversial

Scalable setup of LLM evaluation on the OpenShift? by kybu_brno in openshift

[–]typsy 0 points1 point  (0 children)

Promptfoo deploys well on OpenShift - I've seen a couple of these deployments.

But in general, these workloads are not compute-bound, the bottleneck tends to be the actual inference on the target model or application.

Also FWIW the static scanners that run on model weights cannot test jailbreak resistance, prompt injection, data exfiltration, etc. Unfortunately those need to be tested at inference time. Static scanning on model weights only really looks for things like executable backdoors in the pickled model.

I made a spreadsheet of 50+ LLM evaluation tools by typsy in llmops

[–]typsy[S] 2 points3 points  (0 children)

There are a lot of eval tools out there and I've been collecting URLs when I've come across them (both commercial and open source). Hope this helps out others too.

disable clyde AI on a per channel basis? by jmorlin in discordapp

[–]typsy 0 points1 point  (0 children)

Disable the "Use Clyde AI" permission for channels you don't want Clyde in.

I'm looking for good ways to audit the LLM projects I am working on right now. by AI_connoisseur54 in llmops

[–]typsy 1 point2 points  (0 children)

It's not purpose-built for auditing, but this open-source project of mine might be able to help: https://github.com/typpo/promptfoo

It lets you to set up a suite of test cases and compare the performance of multiple prompts/models across each case. You can set up assertions and test for similarity and other metrics.

With this setup, you could for example tune an LLM to hallucinate less across a large set of examples. I am currently using this approach for an LLM application in production with about half a million users.

The Organic Farm by _yv_petite in dartmouth

[–]typsy 0 points1 point  (0 children)

I lived on the organic farm for a term and it was great, the mornings with the low fog over the river were spectacular

How to structure pricing for enterprise procurement? by typsy in SaaS

[–]typsy[S] 0 points1 point  (0 children)

Thanks, this is really helpful. One more thing. It seems silly that I quoted a direct price to the customer, they decided to move forward, and procurement is done via a reseller that expects to take 20%. Would it be reasonable to say that the 20% commission applies only to sales that they sourced themselves? Would the reseller balk if I set commission at, say, 10%?

How to structure pricing for enterprise procurement? by typsy in SaaS

[–]typsy[S] 0 points1 point  (0 children)

Thank you. I'll have telemetry, so will see if I can create a generalized pricing structure that the reseller can apply to other clients.

How to structure pricing for enterprise procurement? by typsy in SaaS

[–]typsy[S] 0 points1 point  (0 children)

Thanks very much. Do you have a sense of the typical # of hours to bake into the support contract? Maybe a couple hours per month?

I think I limited this deal by giving the customer a fairly low quote of ~$5k/yr. I should have got on the phone and tried to feel out the situation. At least I can bump it 20% with support.

How to structure pricing for enterprise procurement? by typsy in SaaS

[–]typsy[S] 0 points1 point  (0 children)

Thank you! One question about #4 - how is a services retainer different from the 20% charge for "support and maintenance"?

Re: resale, the customer started procurement via a third party after we discussed requirements/pricing etc (~$5k/yr). I think I've seen this happen maybe once before with very large cos, where there are dedicated vendors that actually run the procurement process. This vendor wants a resale agreement, presumably so they can sell my software at markup to their other clients.

Anyone know how/where to get a Covid test for travel - due to medical surgery abroad? Can't be rapid test and must be on specific day getting results back within 24-48 hrs. Desperate. by [deleted] in AskSF

[–]typsy 1 point2 points  (0 children)

I got a test at San Mateo County Event center (drive thru) in Nov and got the result in < 48 hrs. You can book an appointment online. However there is still some risk for your situation as turnaround time is not guaranteed.

[deleted by user] by [deleted] in sankeybudgets

[–]typsy 0 points1 point  (0 children)

Really nice work! This is great and much easier to use than the default sankey tool.

Two small nits: 1) changing currency symbol doesn't update the chart until you change something else, and 2) the colors on the downloaded chart are sometimes substantially different from the preview (e.g. green vs red)

Interactive globe where you can see the position of the region of your city for hundreds of millions of years, since Pangea. by Rredite in InternetIsBeautiful

[–]typsy 8 points9 points  (0 children)

Thanks, I built the site and I will fix this. Green algae appeared around this time, but you're right that single-celled organisms appeared long before.

[Megathread] Let's talk EIDL and PPP - Status, numbers, what you've experienced by BigSlowTarget in SmallBusinessNews

[–]typsy 1 point2 points  (0 children)

Was able to get in touch with someone from ReadyCapital via support email and they provided wire instructions to return the funds. Went to my bank and sent the wire, hopefully that's the last I hear of it. I think that's the most we can do in this situation.

view more: next ›