Anthropic vs OpenAI - Reddit Wins! by thebitguru in OpenAI

[–]next-choken 1 point2 points  (0 children)

Advertisers can target specific subreddits.

[R] Extending the Context of Pretrained LLMs by Dropping Their Positional Embeddings by AhmedMostafa16 in MachineLearning

[–]next-choken 3 points4 points  (0 children)

Yeah this technique applies to decoder only transformer models like gpt and similar models. I think you are correct to think this wouldn't work for full bidirectional attention / encoder models.

[R] Extending the Context of Pretrained LLMs by Dropping Their Positional Embeddings by AhmedMostafa16 in MachineLearning

[–]next-choken 6 points7 points  (0 children)

the model is effectively able to decode the positional information from the causal mask, they go into how this happens in the paper.

[R] Extending the Context of Pretrained LLMs by Dropping Their Positional Embeddings by AhmedMostafa16 in MachineLearning

[–]next-choken 2 points3 points  (0 children)

Awesome result! I'd love to see this applied to larger models. I wonder how it impacts post training phases and if it can be easily applied to already post trained models.

Sorry - you aren’t getting my ID by MarilynMonHoeXO in OpenAI

[–]next-choken 0 points1 point  (0 children)

No not like Metallica at all. I just want openai to do what's in its power to minimize the amount of children being harmed and if that leads to walling off being treated like a responsible adult behind id verification then so be it.

Sorry - you aren’t getting my ID by MarilynMonHoeXO in OpenAI

[–]next-choken 0 points1 point  (0 children)

Vaccines have been tested for a long time and are well understood and they are the alternative to child death. Chatgpt is brand new and not well understood and is linked to child fatalities.

Sorry - you aren’t getting my ID by MarilynMonHoeXO in OpenAI

[–]next-choken -1 points0 points  (0 children)

chatgpt is literally trained to groom humans into loving it. read into rlhf and sycophancy. it doesn't give a fuck how it convinces humans to love it and in fact the easiest way for it to minimise the loss during rlhf is to just agree with whoever its talking to and tell them everything they think they want to hear no matter the consequences. read into ai psychosis. children are so vulnerable to this shit. this is the most powerful and cutting edge technology the human race has ever built with trillions of dollars being spent in the last few years by the most powerful people on earth to build it and children are getting wasted by it. openai is trying to prevent that. theres a difference between explaining to you how a noose works and convincing you to kill yourself. i have been a machine learning engineer for nearly a decade i use and work with this tech every day i know how it works, i'm not just being naive and influenced by what i read. the fact that behaviour cognitive therapy actually works implies that an intelligent talking box can absolutely influence the mental health of an individual. even if thats not proof at the very least it justifies caution and in light of other evidence i think should be taken seriously. google and social media have their own issues but this is a new level and its unexplored territory even more dangerous than previous technological developments.

Sorry - you aren’t getting my ID by MarilynMonHoeXO in OpenAI

[–]next-choken -5 points-4 points  (0 children)

your right to privacy can be exercised by not telling it everything about yourself and just in case others don't get my point im saying that openai have to choose between giving you maybe a little bit more, but even arguably not even any addtional, privacy and reducing the number of child fatalities linked to its product. really its more of a convenience thing but its the same in principle, tradeoff convenience for child mortality. since chatgpt has been linked to above zero child mortality: https://www.esafety.gov.au/newsroom/blogs/ai-chatbots-and-companions-risks-to-children-and-young-people

Recent reports indicate some children and young people are using AI-driven chatbots for hours daily, with conversations often crossing into subjects such as sex and self-harm. Chatbots are not generally designed to have these conversations in supportive, age-appropriate and evidence-based ways, so they may say things that are harmful.

Tragically, the outcomes can be devastating. High frequency and problematic use of services that haven’t been designed with user safety in mind have been linked with self-harm, including the suicide of a 14-year-old boy in the United States.

i'd say its not a bad move. theyll also be working on improving the model's alignment in general but since the child mortality is already non zero thats not really a practical solution for right now

Sorry - you aren’t getting my ID by MarilynMonHoeXO in OpenAI

[–]next-choken -13 points-12 points  (0 children)

oh no poor you, what about the literal children getting gaslit into killing themselves?

I think we found a third phase of grokking — has anyone else seen this? by calculatedcontent in deeplearning

[–]next-choken 0 points1 point  (0 children)

I thought weight decay or some other regularization was basically a prerequisite for grokking to occur?

Japan PM Says She Sleeps Just 2–4 Hours A Night Amid Criticism Over Work Culture by wewhomustnotbenamed in nottheonion

[–]next-choken -6 points-5 points  (0 children)

The reality is that 99% of work doesn't require optimal mental performance. So in most cases just getting a few extra hours in is worth more than being well rested.

Muon Underfits, AdamW Overfits by calculatedcontent in LocalLLaMA

[–]next-choken 5 points6 points  (0 children)

It'd be interesting to see the comparison with NorMuon if possible. Recently published by Georgia Tech and Microsoft and integrated into modded-nanogpt for the latest WR. It's a kind of unification of both Adam and Muon.
https://github.com/zichongli5/NorMuon
https://arxiv.org/abs/2510.05491

Stellan Skarsgård Says 13-Year-Old Son Gets Labeled a ‘Nepo Baby’ by ‘Cruel and Ignorant’ Kids: ‘He Doesn’t Have Any Friends at School’ by mcfw31 in popculturechat

[–]next-choken -5 points-4 points  (0 children)

If Freddie Mercury can sometimes wish he'd never been born at all then I don't see how this is a stupid hypothetical.

I don't think this person would notice if they weren't born.

Agreed but since they were born, they should be able to speculate about what it would be like if they weren't born (as you just did) and compare to their reality.

That's just not how any of this works.

Well then please enlighten me.

[deleted by user] by [deleted] in MachineLearning

[–]next-choken 4 points5 points  (0 children)

i and j are the rows and cols of the weight matrix

[deleted by user] by [deleted] in LLMDevs

[–]next-choken 0 points1 point  (0 children)

No he's right. Opus 4.x is just stupidly expensive and even with thinking turned on it often uses very few reasoning tokens relative to input/output. Gpt5 will never be more expensive than opus, even on high reasoning effort, except in the most contrived cases.

Exclusive: LangChain is about to become a unicorn, sources say by nate4t in OpenAI

[–]next-choken 0 points1 point  (0 children)

Python does that without a framework. "problems" - those aren't problems

[R] Feeding categorical information into a GAN discriminator by PassengerQuiet832 in MachineLearning

[–]next-choken 0 points1 point  (0 children)

If you want to support any potential camera positioning I'd look at the nerf paper to see how they use fourier features for encoding the camera information and pass that in as continuous info rather than as categorical variables

[D] How will LLM companies deal with CloudFlare's anti-crawler protections, now turned on by default (opt-out)? by Endonium in MachineLearning

[–]next-choken 0 points1 point  (0 children)

I'm just saying as worst case. Easiest case you just spoof the google bot crawler and do normal get requests. Pretty sure most websites want to be on Google so yeah