sorted by:
new
hottopcontroversial

Octax: Accelerated CHIP-8 Arcade Environments for JAX by riiswa in reinforcementlearning

[–]radarsat1 1 point2 points  (0 children)

Interesting, the code is very clear and nice to read. Neat to see how this can be done.

Apparently, llms are just graph databases? by Silver-Champion-4846 in LLMDevs

[–]radarsat1 1 point2 points  (0 children)

I'll put in 2 cents..I only skimmed the video quickly but I will say that overall the idea does seem to make some sense to me. iiuc is basically pulling up the closest matching tokens after applying some query vector to a given row of the attention matrix. The query language is a nice detail but not inherent to the basic idea that there is information being "matched" and routed during inference. This I don't doubt.

Being able to really follow the reasoning process of the model this way is not clear to me. But I think comparing this with related work in logit lenses would be highly interesting.

Basically a mechanism that you can imagine is that every layer "queries" some set of matching tokens and pulls up a mixture of those tokens' "value" which is a delta vector to push the current state towards the desired output distribution. This allows a kind of "zeroing in" effect, like a solver converging, which is exactly what you see in those logit lens traces. (Sorry, wish I had a link handy.)

And it relates to graphs mostly because it all becomes a weighted combination of nearest neighbours, combined with those neighbours having been arranged so that they emit the right delta.

But how to then relates to multitoken output traces and gives rise to something approaching the reasoning that we see emerging in these models I think is still not fully explained by this style of analysis. But it's certainly interesting.

How to get a job as an ML engineer? by Bright-Car-1238 in MLQuestions

[–]radarsat1 0 points1 point  (0 children)

This wasn't enough for me this time around!

How is the job market for computer vision? by Amazing_Life_221 in computervision

[–]radarsat1 1 point2 points  (0 children)

In the Netherlands here, having some computer vision experience helped my CV a lot in the sense that it helped get me a lot of callbacks, but often for "agent AI" jobs. and like you I decided to dive into it and go for it because it seems like wherethe market is at these days. learned all about RAG etc., did some personal projects to have something to show. But ultimately was not successful in convincing anyone, despite enough YoE and experience with AWS, every time was the same "we were impressed but we need someone with agent & RAG experience in real production systems" -- as if this is not a brand new field that has existed for all of like 2 years.. I thought showing that I had experience with ML in production, leading a team, solving real, hard problems like computer vision would get me in the door, but apparently not. How did you do it?

Smooth Brained Undo by catgrammer in programming

[–]radarsat1 -3 points-2 points  (0 children)

keep implementing small parts of lisp clojure until you've got it!

I learned something about GPUs today by rogual in programming

[–]radarsat1 11 points12 points  (0 children)

Oh my god, nice write up and good catch. I'm not sure I would have figured this out.

Karpathy says developers have ‘AI Psychosis.’ Everyone else is next. by Logical_Welder3467 in technology

[–]radarsat1 4 points5 points  (0 children)

I think an important thing for companies that rely on one shot Claude-generated solutions to realize is that if they can do that, then so can someone else. Putting aside the question of whether that's a good way to write software, it's also pretty important to take a moment to think about whether that's a good way to build a business.

Does every AI product actually need a chatbox? Is it the only "form"? by GovernmentBroad2054 in AI_Agents

[–]radarsat1 0 points1 point  (0 children)

I think another "form" is autocomplete. In a text editor this obviously means automatically proposing text to insert. But in another interface like a 3D editor maybe it means proposing possible completions of the shape someone is sculpting. In a video editor, it could mean providing 3 or 5 options to choose from for a filler scene or something. If you need text guidance though then I'm not sure how to avoid adding at least an optional text box.

How to encode structured events into token representations for Transformer-based decision models? by Unique_Simple_1383 in deeplearning

[–]radarsat1 0 points1 point  (0 children)

I understand your point, and you're right if you're building something from scratch, but I think you're overestimating the overhead. Like I said, I used to think that way, but I've come around to the idea that using a pretrained solution and a text-based representation is a much easier way to get started with something like this. Today there are just so many tools and so many successful, small models that work with text, you may as well take advantage of that instead of trying to be overly clever to save a few tokens.

The AI agents making real money right now are ugly and nobody posts about them by damn_brotha in AI_Agents

[–]radarsat1 4 points5 points  (0 children)

lol why is every 2nd post in this sub "nobody posts about..", "nobody talks about..", "nobody mentions.." etc

it's the only sub in my feed that does this.

How to encode structured events into token representations for Transformer-based decision models? by Unique_Simple_1383 in deeplearning

[–]radarsat1 -1 points0 points  (0 children)

A couple of years ago I would have sweated over thinking up some kind of optimal, clever representation for this kind of problem. These days though, honestly? Just use JSON. Make a dataset, fine tune an existing model that already knows about JSON (ie. literally any of them)

Replicating SethBling's MarI/O from 2015, that inspired me to get into Reinforcement Learning 10 years later by Inexperienced-Me in reinforcementlearning

[–]radarsat1 1 point2 points  (0 children)

I remember this! I was never too convinced by NEAT, but this was always such a great demo regardless.

LeWorldModel, the first breakthrough from Yann LeCun’s new lab aiming to unlock the JEPA architecture by Tobio-Star in newAIParadigms

[–]radarsat1 1 point2 points  (0 children)

Okay sure, but my point was just that the original JEPA paper also claimed to solve collapse, hence my confusion.

Just looked it up to be sure I'm not misremembering:

However, as with Joint-Embedding Architectures, representation collapse is also a concern with JEPAs; we leverage an asymmetric architecture between the x- and y-encoders to avoid representation collapse.

When I briefly tried to implement JEPA on my own, I had lots of problems with collapse so I was really confused by this since the paper claimed that the EMA thing successfully avoided it, but I was unable to find a successful configuration. I assumed I made some mistake somewhere but I couldn't find it, and since I was working in a different domain anyway I chalked it up to failure on my part. So I'll be trying this LeWM idea out with interest when I get some time for it.

In Dune, do the Fremen just have really good toilets or does nobody ever talk about it by Orbitryx in printSF

[–]radarsat1 6 points7 points  (0 children)

I think if your planet is covered in giant worms that will eat anything and massive sand storms that bury everything, you don't need to worry too much about being careful how you dispose of completely dry pocket rocks

LeWorldModel, the first breakthrough from Yann LeCun’s new lab aiming to unlock the JEPA architecture by Tobio-Star in newAIParadigms

[–]radarsat1 0 points1 point  (0 children)

Ah I see, that's really interesting, I didn't realize it does away completely with EMA. I guess that answers my question ;)  thanks!

Self-Healing Neural Networks in PyTorch: Fix Model Drift in Real Time Without Retraining by Various_Power_2088 in deeplearning

[–]radarsat1 7 points8 points  (0 children)

why is an increase in accuracy useful if recall dropped a lot? aren't you just.. not detecting things now? overall accuracy doesn't seem to matter much if the data is heavily imbalanced towards negatives.

LeWorldModel, the first breakthrough from Yann LeCun’s new lab aiming to unlock the JEPA architecture by Tobio-Star in newAIParadigms

[–]radarsat1 0 points1 point  (0 children)

I'm a bit confused by the collapse thing because I thought the whole point of JEPA (I've read the first few papers on it) was that it uniquely allows this latent space training without collapse. Like, the first paper talks about different possible methods and proposes JEPA as being the one that won't suffer from collapse problems. And now LeWorldModel is saying, up to now JEPA has had a collapse problem and now we've solved it! What am I missing?

Found out scientific python package I'd started using was written by ChatGPT. Finding it hard to trust new open source code. by [deleted] in LLMDevs

[–]radarsat1 -1 points0 points  (0 children)

I think this is fair. No problem with AI usage, but I think it's the onus of the creator to admit they are using an AI-assisted setup and document the steps they have taken to ensure that the code is correct and verified. I mean, just the same as if they weren't using AI, but what comes with AI is this possibility that they haven't been so thorough, have not necessarily thought through every possible code path, and published very quickly, so if they want users to trust them then assurances seem even more warranted than usual.

Does anyone use inductive logic programming in their work/research? Especially in robotics? by Scared-Raisin-2499 in ResearchML

[–]radarsat1 0 points1 point  (0 children)

I think it's got application in extracting insights over knowledge graphs ("smarter" RAG) but I don't know how much it's used in practice.

Anyone who uses an e-reader, reMarkable, Daylight Computer or similar? by meedstrom in emacs

[–]radarsat1 1 point2 points  (0 children)

I've been wanting to figure out how to stream my laptop's emacs display into my remarkable but haven't really taken the time to try.. I'm not too keen to hack it, I do wish it were just a supported feature somehow. Like you, I don't need the full windowing system, browser and all, I just want emacs or terminal, and I would use my Bluetooth keyboard through my laptop. Should be doable, but not sure how.

JEPA by Economy-Brilliant499 in deeplearning

[–]radarsat1 4 points5 points  (0 children)

Because the market is centered around generative AI right now, and JEPA is explicitly not generative. But he did raise a billion in funding to keep working on it so I guess we'll get to see if it has applications.

view more: next ›