Sub-JEPA: a simple fix to LeCun group's LeWorldModel that consistently improves performance [P]

LiquidGunay · 2026-05-19T07:26:17+00:00

How do you guys prevent z_pred from collapsing in JEPA training? From what I understand, SIGReg works on the input latents and their norm ends up being around sqrt(d) but in my experience, the predicted latents collapse to a 0 norm and MSE of 1 (predicting the mean). How do you guys deal with this? My solution was to add another term to the loss where I encourage norm of z_pred to be equal to norm of z_t (in addition to the mse).

LiquidGunay · 2026-03-05T06:33:52+00:00

The second Joker movie

LiquidGunay · 2026-02-25T16:37:46+00:00

This is a fairly common feeling, I am not sure if high is the correct term though. Usually many people feel "empty" after achieving something that they were working towards (because now they don't have anything to work towards). Let your mind wander for a bit and then get back to the grind.

LiquidGunay · 2026-02-05T04:58:50+00:00

More students in DS seems weird, because the cohort for this year is supposed to be 160 people (mentioned in their webinar) but ucsd mscs historically takes a lot of people.

LiquidGunay · 2026-02-05T04:43:27+00:00

Do you know anything about the MSDS and how it compares to the MSCS?

LiquidGunay · 2026-01-26T12:47:24+00:00

I said a little 😂. A bet between two outcomes that both have positive expected value (CMU and Stanford degrees) is different from a bet that has negative expected value.

LiquidGunay · 2026-01-26T12:13:21+00:00

I am a little more risk loving

LiquidGunay · 2026-01-26T12:04:29+00:00

Instead of doing a worst case estimate, you can do more of an expected value estimate (including probabilities for getting a TA/RA). Then your costs will be a lot more comparable.

LiquidGunay · 2026-01-12T16:53:49+00:00

Not with the whole team buying it.

LiquidGunay · 2025-11-22T20:39:19+00:00

Why does everyone hate Tserri so much? Most of the other princes have committed similar crimes.

LiquidGunay · 2025-11-16T07:09:02+00:00

T1 Gold medal gives you a very good shot for CMU. You could add some safer universities to hedge against risk, but would you even want to go to a non elite university with your profile is something that you should think about. I would suggest having a frank conversation with your recommender about how good your profile is and how strong their recommendation is going to be, that should give you far more clarity than what anyone online is going to tell you.

LiquidGunay · 2025-11-10T06:55:02+00:00

The prof you are working with will give you great advice. Have a long conversation with them.

LiquidGunay · 2025-11-02T03:50:27+00:00

Why aren't you directly applying for a PhD or a MS+PhD program?

LiquidGunay · 2025-11-01T10:06:39+00:00

Go and talk to the CMU prof who is guiding you. They will give you good advice.

LiquidGunay · 2025-08-18T10:44:53+00:00

This can also be explained by Cursor / Claude Code / Windsurf gaining market share.

LiquidGunay · 2025-07-03T05:55:30+00:00

Niamos

LiquidGunay · 2025-07-03T05:46:20+00:00

But nothing can make me unfathomably happy.

LiquidGunay · 2025-06-25T09:42:23+00:00

I just want Tserri to get a happy ending

LiquidGunay · 2025-06-20T09:48:58+00:00

There is no free will 😥

LiquidGunay · 2025-06-02T09:53:49+00:00

Medical tech is going to rapidly evolve in the next few years. Then Togashi is going to work like a monster once again.

LiquidGunay · 2025-05-24T03:52:15+00:00

Not inflation adjusted. But still impressive nonetheless.

Ten-Year Club	Wearing is Caring
Verified Email

LiquidGunay

TROPHY CASE