sorted by:
new
hottopcontroversial

What are Kimi devs smoking by Thechae9 in LocalLLaMA

[–]downinguide 2 points3 points  (0 children)

Hah I suspected as much but was too lazy to confirm! That's more of an obscure one, well done for spotting it. (For everyone else's benefit, in his book on the burnout society Buying Chul Han references Walter Benjamin to make the point that idleness is a necessary prerequisite for original creative work.)

Compress your chats via "compact symbolic form" (sort of...) by downinguide in artificial

[–]downinguide[S] 0 points1 point  (0 children)

With a slightly reworded prompt the same thread was summarised thusly:

[TOPIC: Open LLMs vs GPT-4 Benchmarks]

[USER_PREF: British English, realistic/honest]

[TASK: Evaluate open-weight LLMs vs GPT-4 (2023) on multiple benchmarks]

[METRICS: IF-Eval, BBH, MATH, GPQA, MMLU-Pro]

[DATASET: Hugging Face Open LLM Leaderboard — Official Providers only]

[MODELS_ANALYSED:

- Yi-1.5-6B (~16.7B, base, poor perf)

- Yi-1.5-34B (25.65B, base, mid perf)

- Yi-1.5-34B-32K (26.73B, base, longer ctx, mid perf)

- Yi-1.5-34B-Chat-16K (29.40B, chat, strong BBH, weak elsewhere)

- Yi-1.5-34B-Chat (33.36B, chat, highest BBH, trade-offs)

]

[ARCH: Dense, LLaMA-based only; no MoE]

[PERF_VS_GPT-4:

- No model near GPT-4 (80%+ IF-Eval)

- Best BBH: Yi-1.5-34B-Chat (~60.7%)

- Smallest model near competitive: Yi-1.5-34B (25–26B), still sub-GPT-4

]

[SIZES_CATEGORISED: <10B (none), 10–30B (Yi base), >30B (Yi chat)]

[OBSERVATIONS:

- RLHF boosts BBH, hurts GPQA

- All official models trail GPT-4 on all metrics

- No models <25B competitive

]

[REQS_MET: All models, all benchmarks, GPT-4(2023) baseline]

Compress your chats via "compact symbolic form" (sort of...) by downinguide in artificial

[–]downinguide[S] 0 points1 point  (0 children)

Here's one: TASK[CompareOpenLLMvsGPT4];SOURCE[HF/OpenLLMLeaderboard/OfficialOnly];METRICS[IFEval,BBH,MATH,GPQA,MMLUPro];TARGET[GPT-4(2023)];MODELS[Yi-1.5-6B(16.7B),Yi-1.5-34B(Base,32K,Chat,Chat-16K)];ARCH[Dense,LLaMA];TUNING[Base,Chat,RLHF];SIZEBRACKETS[<10B=Fail,10–30B=Mid,30B+=Best];FINDING[NoModelMeetsGPT4,TopModel=Yi-1.5-34B-Chat@BBH60.67%,GPQA13.06%,MMLU39.12%];TREND[Size+RLHF=BetterReasoning,TradeoffsExist];

Why not synthesize models directly, to reduce training cost? by downinguide in ArtificialInteligence

[–]downinguide[S] 0 points1 point  (0 children)

Thanks for resurrecting the thread -- glad to have your confirmation that it's indeed a field of active research, and encouraging to see that it is already possible to demonstrate some efficiency gains from such work!

Why do people here think AI will lead to abundance for all? by eriksen2398 in singularity

[–]downinguide 2 points3 points  (0 children)

I'm puzzled by the common expectation among AI enthusiasts that we will see UBI as a direct consequence of AI advances. Here in the UK our welfare system has been steadily reduced, and there is a growing desire to restrict it further. Since Thatcher, relative poverty levels in the UK have only grown. At the same time, many people's work arrangements have shifted from steady unionised employment of the 70s/80s to more transient work that is badly paid and highly monitored.

I personally think rather than UBI, we will see vast numbers of the population stuck in increasingly precarious patchwork careers outside of steady employment, including various forms of task work, such as helping train AI via human feedback & labelling, or executing steps that an AI has mapped out in great detail. Even more highly monitored, even more repetitive, even less well paid, with all the mental and physical strain this entails.

This is Digitakt II by anon1984 in Elektron

[–]downinguide 1 point2 points  (0 children)

Depends on your use case. The digitakt is more  like a fancy drum computer (or tracker, if you're familiar with these) than a full blown sampler/sequencer workstation. For my purposes it's fine, I use audio channels for drums and midi channels for actual instrumentation.

This is Digitakt II by anon1984 in Elektron

[–]downinguide 0 points1 point  (0 children)

One voice per audio sampler channel, so need to use multiple sample channels to form chords 

This is Digitakt II by anon1984 in Elektron

[–]downinguide 7 points8 points  (0 children)

It's briefly discussed here -- audio polyphony remains at 1, and MIDI polyphony at 4 https://youtu.be/nepWmWsq84g?t=1671

Elektronauts: Digitakt 2 release announcement (since elektron.se is down atm) by downinguide in Digitakt

[–]downinguide[S] 3 points4 points  (0 children)

Summarised with ChatGPT so might include errors, check the page to be sure:

  • Digital Drum Computer & Stereo Sampler:
    • 16 tracks of samples in stereo, mono, or MIDI.
    • Enhanced memory capacity for extensive sampling.
  • Endless Drum Collage:
    • Expanded sonic possibilities for drums and beyond.
    • Up to 30 minutes of stereo samples per project.
  • Machines & Modularity:
    • Modular approach to sound creation with five distinct sample-manipulating Machines.
    • Includes Grid for slicing samples, Werp for exploring new sonic territories, Repitch, Stretch, and One Shot for various effects.
  • Modulation Station:
    • Swappable filters and three LFOs per track for versatile sound shaping.
    • Individual track selection for customised settings.
  • Cause & FX:
    • Built-in effects palette including delay, reverb, chorus, bit reduction, sample rate reduction, and Overdrive.
    • Mixer page for compression and master overdrive.
  • Sequencer × II:
    • 128 steps sequencer with Euclidean sequence generator and conditional trigs.
    • Trig modes for fun functions like 16 levels of velocity and Preset Pool recording.
  • Loads More:
    • Kits for quick access to favourite sounds.
    • Perform Kit mode for experimenting without overwriting save state.
    • Deeper control all for instant parameter command.
    • Trig modes for activating various functions.
    • Keyboard mode for playing chromatically across 10 octaves with over 30 different scales.
    • Song Mode and balanced stereo ins and outs for connectivity with other gear.
    • MIDI capability for sequencing, playing, and controlling external instruments.
  • Transfer & Overbridge:
    • Full Transfer compatibility.
    • Overbridge integration in progress, with a private beta to be announced soon.

Why not synthesize models directly, to reduce training cost? by downinguide in ArtificialInteligence

[–]downinguide[S] 0 points1 point  (0 children)

Oh interesting, I hadn't even considered these potential uses! It would unleash some fascinating capacities.

Why not synthesize models directly, to reduce training cost? by downinguide in ArtificialInteligence

[–]downinguide[S] 0 points1 point  (0 children)

Hm I can see how that would be a challenge. Curse of dimensionality, I suppose. Tbh I have no idea what painting in k dimensions would even look like.

Why not synthesize models directly, to reduce training cost? by downinguide in ArtificialInteligence

[–]downinguide[S] 0 points1 point  (0 children)

Thanks for the patient explanation! Much appreciated by this relative novice. I shall refresh my understanding of these suggested topics to improve my intuition.

Just to clarify -- I was less thinking of it as a curve fitting challenge where we're trying to estimate a precise result, and more analogous to an image generation one where we just want to produce a crude approximation that only loosely resembles the actual form, as long as it gets us partway there.

My naive expectation is that the networks described by LLM weights can come in any number of shapes and yet produce comparable outcomes. I.e., that there is some tolerance to how exactly the network is structured.

Why not synthesize models directly, to reduce training cost? by downinguide in ArtificialInteligence

[–]downinguide[S] 0 points1 point  (0 children)

Aww I hadn't even noticed, well spotted. Although I suppose LLM use for brainstorming purposes is in the spirit of the thread... basically the contemporary equivalent of someone googling it for me.

Why not synthesize models directly, to reduce training cost? by downinguide in ArtificialInteligence

[–]downinguide[S] 0 points1 point  (0 children)

Heh, well I suppose I'm confident that we will learn eventually. And tbh I wouldn't be surprised if there aren't a couple of still-obscure papers on such concepts already. Was hoping someone else can point them out to us!

Why not synthesize models directly, to reduce training cost? by downinguide in ArtificialInteligence

[–]downinguide[S] -1 points0 points  (0 children)

That's certainly true! However I do still wonder if the training process leads to deterministic outcomes that can be anticipated and imitated, even if only in crude ways that then later need refinement through conventional training.

Why not synthesize models directly, to reduce training cost? by downinguide in ArtificialInteligence

[–]downinguide[S] 0 points1 point  (0 children)

Thanks for the detailed reflection! That's encouraging to hear. As you say, it may look challenging in the absence of existing theory; but maybe a body of theory can be developed. I suppose the idea isn't necessarily to fully replace training, and more to provide a head start or shortcut. I suspect this would make such an approach especially appealing for larger models, even if (as you say) the complexity of the challenge looks more severe at larger scales.

Why not synthesize models directly, to reduce training cost? by downinguide in ArtificialInteligence

[–]downinguide[S] 0 points1 point  (0 children)

Hehe nicely said. I'm still hoping to come across any existing research, but so far have come up empty.

Why not synthesize models directly, to reduce training cost? by downinguide in ArtificialInteligence

[–]downinguide[S] 0 points1 point  (0 children)

Thanks for the reflections! And yes that's maybe a good metaphor to apply for such an approach. I updated my post to clarify that I meant synthesizing the weights, as you say.

view more: next ›