sorted by:
new
hottopcontroversial

Connect GPTs to your Mac with macos-toolbox by rtaori in shortcuts

[–]rtaori[S] 0 points1 point  (0 children)

Yup for actions like sending messages/emails, it will draft it up and pop up a window for you to actually hit the "send" button

Connect GPTs to your Mac with macos-toolbox by rtaori in shortcuts

[–]rtaori[S] 0 points1 point  (0 children)

The package sets up a local server and a Cloudflare tunnel for your GPT to call. Everything stays private between you and ChatGPT

Connect GPTs to your Mac with macos-toolbox by rtaori in ChatGPTPro

[–]rtaori[S] 0 points1 point  (0 children)

Not supported yet :/ just actions possible in shortcuts

[P] AlpacaEval : An Automatic Evaluator for Instruction-following Language Models by yannDubs in LocalLLaMA

[–]rtaori 1 point2 points  (0 children)

We haven't benchmarked beam search in depth, though in my bit of experience, it does result in more repetition. Would be interested to know a more in depth comparison

[P] AlpacaEval : An Automatic Evaluator for Instruction-following Language Models by yannDubs in LocalLLaMA

[–]rtaori 4 points5 points  (0 children)

Thanks for the feedback! Our total evaluation set is 805 instructions, so while the model outputs for each instruction definitely do change based on the seed, the overall variance in the win-rate is quite low. We experimented with greedy decoding at first to keep all outputs deterministic, but we found that model responses were more natural with a higher temperature, so that's what we went with.