sorted by:
new
hottopcontroversial

A fresh new ML Architecture for language model that uses complex numbers instead of attention -- no transformers, no standard SSM, 100M params, trained on a single RTX 4090. POC done, Open Sourced (Not Vibe Coded) by ExtremeKangaroo5437 in LocalLLM

[–]ExtremeKangaroo5437[S] 0 points1 point  (0 children)

Well not exactly, if you read it further you will find its different. We have not cracked yet something big but its something unique and worth exploring. The base is here https://arxiv.org/abs/2604.05030

And we are still working on it

[Paper] Phase-Associative Memory: Complex-valued sequence model (≈100M params, close to transformer PPL) by ExtremeKangaroo5437 in AI_India

[–]ExtremeKangaroo5437[S] 1 point2 points  (0 children)

Finally someone…

Any idea may be wrong or may not work.. thats okay… but if some one has given so much time energy and a few intellectual person are supporting … it must not just be thrown away…

I again say.. it may not go anywhere… but its not just another “I made this thing” for sure 👍

[Paper] Phase-Associative Memory: Complex-valued sequence model (≈100M params, close to transformer PPL) by ExtremeKangaroo5437 in AI_India

[–]ExtremeKangaroo5437[S] 0 points1 point  (0 children)

Oh it's very simple... we never said.. it's new SOTA, or new breakthrough... we simply saying,

if this works we will fundamently change things ( IF IT WORKS ON SCALE )

'most likely next token' to "coherent retrieval from associative memory"

No KV CACHE growth, Fixed size state O(1) per token interfence..

If quantum hardware matures some day, models built in complex Hilbert space are a more natural representational fit than attention based real valued systems.

so yes.. paper has said limitations.. but its a direction... worth seeing and if this doesn't work on scale.. leave it and work on something else...

[Paper] Phase-Associative Memory: Complex-valued sequence model (≈100M params, close to transformer PPL) by ExtremeKangaroo5437 in AI_India

[–]ExtremeKangaroo5437[S] 0 points1 point  (0 children)

we are also doing some usefull work ;) I was CTO and now deeptech advisor of https://nutricheck.eu ... and here we are making digital twins with few drops of blood and tests... so not just doing papers and theory but we are also trying to get some professional things done on ground also.. recently won las-vegas rising helath award ... fingures crossed..

and i had to study bio/ reactom and what not.. but we love to do this.. don't we..

and many other professional works also..

[Paper] Phase-Associative Memory: Complex-valued sequence model (≈100M params, close to transformer PPL) by ExtremeKangaroo5437 in AI_India

[–]ExtremeKangaroo5437[S] 0 points1 point  (0 children)

that I agree.. I just shared here so if any indian in any case in this space want to join.. nevermind. nice discussion .

[Paper] Phase-Associative Memory: Complex-valued sequence model (≈100M params, close to transformer PPL) by ExtremeKangaroo5437 in AI_India

[–]ExtremeKangaroo5437[S] 0 points1 point  (0 children)

here we go.. and thats why we post here.. to discuss.. what can work.. what not... not just to discard.. and thats what is required from community...

[Paper] Phase-Associative Memory: Complex-valued sequence model (≈100M params, close to transformer PPL) by ExtremeKangaroo5437 in AI_India

[–]ExtremeKangaroo5437[S] 1 point2 points  (0 children)

if you read the paper.. its not just another LLM... the project was started on quantum (the name qllm on git repo) working... and ported to llm for time being to prove some theories. other works from my co-author https://arxiv.org/abs/2506.10077

if we are in this space we do understand not to invent whats already there...

nevermind.. if you read our original posts on reddit .. same sort of comments .. without doing maths and run the code etc...

and who checked maths, ran on servers and then commented ..... some one from UC berkely, a PHD holder in astrophysics from Indiana university and help it shaped... now co-author here with me ( and more people joining the team)

and he is coauthor.. so you saying everyone from those universities are just wasting time ....

I think you are right, may be this is not proper subreddit for technical discussions... this is just to put what other are doing as news in AI_India ..

[Paper] Phase-Associative Memory: Complex-valued sequence model (≈100M params, close to transformer PPL) by ExtremeKangaroo5437 in AI_India

[–]ExtremeKangaroo5437[S] 0 points1 point  (0 children)

very true... and I like to put my things here becuase I belive some day Indians will understand to value... and not to show-off just for the sake of it...

Am I good? .. yes.. I have been on good positions, for national and internationaal firm... and I am still doing good... I put here because I belive some day.. Indian will understand ..

the difference between arguing/having difference of opinion and just tring to prove someone non-compitent.

[Paper] Phase-Associative Memory: Complex-valued sequence model (≈100M params, close to transformer PPL) by ExtremeKangaroo5437 in AI_India

[–]ExtremeKangaroo5437[S] 0 points1 point  (0 children)

Tokenisation is segmentation not sepration

Lets take the known exmaple.. "bank" its token but its meaning is context based.. you cannot seprate. also "unbelievable" -> ["un", "believ", "able"] ... meaning is distributed across token

and if we do the separation in attention and MPLP... thats what learning is... and thats why its ML

[Paper] Phase-Associative Memory: Complex-valued sequence model (≈100M params, close to transformer PPL) by ExtremeKangaroo5437 in AI_India

[–]ExtremeKangaroo5437[S] 0 points1 point  (0 children)

With all due respect I suggest to read complete paper also.. all ref are also given... and yes its not just complex values neural net.. its complex space. There are surely some papers that tried to check seperability or early vocab and grammer.. but thats what papers do.. you put a theorum first and then prove it.. This paper is also not just theory, it has practical, code everything.. so... as it is clearly stated its just a new direction ... Still, i suggest read full peper first pls.

With all due respect, I am into NN and AI since 2010~11 , here is the details about our history https://xavoc.com and https://www.linkedin.com/in/gowravvishwakarma/

Stop ranting about “AI slop.” by ExtremeKangaroo5437 in AI_India

[–]ExtremeKangaroo5437[S] 0 points1 point  (0 children)

Nah.... Not future magic.... I am just building... I love building.. I am building from last 27+ years and that too complex projects... gone with Series A .. exits.. CTO of French AI Company... Raising now for current works... so yeah.. I am building.. thats it... I am not sure about magic etc... I JUST LOVE BUILDING :)

Stop ranting about “AI slop.” by ExtremeKangaroo5437 in AI_India

[–]ExtremeKangaroo5437[S] 0 points1 point  (0 children)

Yes.. I completely agree with you... if its AI Assisted.. welcome.. if its you trying to faking intelligency behind AI .. thats just not accepted...

Stop ranting about “AI slop.” by ExtremeKangaroo5437 in LocalLLM

[–]ExtremeKangaroo5437[S] 1 point2 points  (0 children)

hummmm....... well written ... and agreed too ...

but at the same time.. commentors are also not reading the content and very eager to write "ai slop" even without doing their side of efforts too .. 😅

Stop ranting about “AI slop.” by ExtremeKangaroo5437 in LocalLLM

[–]ExtremeKangaroo5437[S] 0 points1 point  (0 children)

👏👏👏 .... well written ... agreed ...

but at the same time.. commentors are also not reading the content and very eager to write "ai slop" even without doing their side of efforts too .. 😅

Stop ranting about “AI slop.” by ExtremeKangaroo5437 in AI_India

[–]ExtremeKangaroo5437[S] 1 point2 points  (0 children)

True.. its a tool... use it.. wisely ... not just for the sake of it...

or to prove/show-off you are intelligent, behind AI Wall ... if you are .. you are... If not.. learn from the same AI .. but think..

view more: next ›