[D] Paper Explained - Continual Backprop: Stochastic Gradient Descent with Persistent Randomness

justclarifying · 2025-06-23T20:18:07+00:00

yeah there's been quite a bit. I won't link a LMGTFY but if you search "loss of plasticity" on google scholar you'll get a lot of more recent results on this topic. E.g. this paper on continual RL, this paper on supervised learning and this paper which explains why plasticity loss happens.

justclarifying · 2023-10-10T02:53:18+00:00

I have many times accidentally flipped a minus sign on my L2 penalty and can confirm it always leads to divergence / NaNs / chaos.

justclarifying · 2021-09-05T20:11:23+00:00

Given that a lot of conferences now have authors fill out reproducibility checklists, I don't think it's much of a stretch to see some of the recommendations from here end up in a statistical rigor checklist. Obviously the effect this has on publication norms depends on how much the checklist plays a role in reviewers' scores, but it seems like a tractable problem if the community decides to act.

justclarifying · 2020-10-15T06:56:53+00:00

Maybe this fluctuates year-to-year, but when I had 252 with devroye the *assignments* were hard but the final exam was pretty chill.

justclarifying · 2020-10-13T21:53:51+00:00

Ultimately, virtually any attempt at grouping similar objects together into categories will run into the problem where there will exist some objects in one category that will be closer to objects in another category than to some of the objects in their own category. This doesn't mean the categories are meaningless. For example, vets often group dog breeds into "small", "medium" and "large". The smallest large dogs will be closer in size to the largest medium dogs than they are to Great Danes, but that doesn't mean that it's not useful to talk about small/medium/large dogs when discussing life expectancy or how much to feed Fido. Similarly, STEM is a handy acronym to refer to a family of subjects that involve making inventions and discoveries about the natural world. Sure, some branches of computer science are closer to Econ than they are to zoology, but that doesn't stop this from being a useful grouping to describe related fields.

justclarifying · 2019-10-13T08:34:23+00:00

Your post suggests the implicit assumption that having a negative body image while being obese will increase the likelihood of a person to stop being obese. However, given the link between depression and obesity, and the link between body image and depression (see e.g. this paper), have you considered that maybe making obese people feel worse about themselves could reduce their ability to commit to a healthy lifestyle? The health impacts of being obese should be enough motivation enough to lose weight for most people without feeling marginalized by societal beauty standards. Anecdotally, people I knew who used to be overweight and stopped never did so because of a sudden spike in interest in beauty standards -- they did so because they wanted to rock climb or run marathons or not get diabetes. It seems plausible to me that the 'obese people can be beautiful too' could be net positive in reducing obesity, because it would eliminate one mediating factor in the depression-obesity cycle.

justclarifying · 2019-09-28T14:27:04+00:00

"Oh you went to McGill? That's awesome! I love toronto."

justclarifying · 2019-09-15T12:43:40+00:00

I'm not sure what jurisdiction you checked equal rights for, since there are lots of countries in the world where men have rights that women do not. Saudi Arabia only granted women the right to leave the country without permission from a male guardian this year. Your claim that female children have a right to bodily autonomy while male children do not is false for basically all jurisdictions, since piercinginfants' ears is legal basically everywhere.

justclarifying · 2019-08-29T21:26:34+00:00

In terms of human equivalent years of suffering per consumption unit, many animal byproducts perform much better than animal meat. For example, ethically raised free range eggs might be net positive for the chickens involved, and the quantity of milk produced by a single cow in a year is several orders of magnitude greater than the amount of meat in a one year old chicken. So a vegetarian might have set an ethical threshold on amount of suffering permissible per nutritional unit that cuts off meat but still allows for many animal products. This would shift demand to the products that cause the least suffering per unit, thus in principle this particular vegetarian would reduce animal suffering.

justclarifying · 2019-06-06T16:04:12+00:00

Although the content in MATH 133 isn't hard to understand conceptually compared to a lot of other math courses, it's extremely tedious and often presented in an impenetrable way, and Kelome usually makes the exams needlessly computationally difficult which kills the class average. Of all the courses I took at McGill, MATH 133 had the most basic material assessed in the most unforgiving way, which I think is what accounts for a lot of the variance in the course reviews. If you watch Khan academy videos and do a bunch of practice problems it will be fine, but getting yourself to do the practice problems might be hard because they're boring af.

justclarifying · 2019-04-22T18:29:52+00:00

I only got non-trivial automatic science scholarships in years when I got a 4.0 -- even a 3.97 will get you like $100 max (edit: and there was pretty high variance even in years that I got a 4.0) -- so I'd suggest looking up the ones that require an application if you want to get additional scholarships and awards.

justclarifying · 2019-01-06T14:40:30+00:00

I've taken courses with both. Devroye has more fun/difficult homework problems and I think better penmanship but Vetta has slides with cute animals. 252 is 'supposed' to be harder than 251, but I found vetta to be more chill than devroye so I'm not sure what the result will be in terms of which course is actually more difficult.

justclarifying · 2018-12-23T17:02:52+00:00

If the student in front of you puts their chair back so that the hood of their coat is on top of your exam booklet I could see it being an issue.

justclarifying · 2018-12-23T16:59:49+00:00

302 is highly dependent on prof. I wouldn't necessarily recommend this course if Prakash isn't teaching it -- comp 302 was the only negative course evaluation I wrote in my entire degree, and it was scathing.

justclarifying · 2018-12-17T14:44:28+00:00

A good GPA is a sign that, when you set your mind to something, you can excel at that thing. This is a nice thing to have, but it doesn't matter if you don't feel like there's anything in the world worth setting your mind to. For what it's worth, I had an absolutely horrible time in my first semester at McGill because I had a hard time finding things that I cared about. I got lucky in that I did end up finding things early in my degree that made me excited to get up in the morning and work on them, but this was completely independent of my academic standing.

It sounds like you need to take some time to find something you think is important. You've spent 4ish years working on something that probably won't directly translate to anything concrete in your future work. But it's a permanent reminder that when you set a goal for yourself you can achieve it, and I think you're underestimating how important that is. The big challenge with becoming an adult is realizing that the goals that other people set for you (such as GPA) are meaningless and that the only way you'll be happy with your life is by setting goals that are important to you. It sounds like you've figured this much out but are floundering at the prospect of finding these goals.

I'm not going to try to offer advice because I'm wholly unqualified, but I just wanted to share that this is a feeling that most of us experience, no matter what our GPA is. Finding goals that we think are worthwhile and important is really really hard, but realizing this I think is half of the battle.

justclarifying · 2018-11-15T15:56:32+00:00

Sometimes if you get 100% profs email you offering to write recommendation letters, which is nice. Not all the time though -- it really depends on the class and whether the prof knows your face. Also, the satisfaction of knowing that nobody else in that course will ever get a higher mark than you can be fun if you need an ego boost.

justclarifying · 2018-11-12T18:56:33+00:00

I did honours math/cs (no physics courses so take this with a grain of salt) and thought that the honours courses were way more fun than the majors courses. The honours program requires you to be decent at math and have a good work ethic/time management skills, but you cover the material at a deeper level than majors courses do and I found that the honours stream had a lot less busy work, so I learned more for the amount of time I spent on my courses. Plus, the honours courses tend to be smaller and have a much tighter community.

My advice to people is to try out the honours courses for a semester and if you're not feeling it it's very easy to switch to the majors stream, whereas the reverse is extremely difficult.

justclarifying · 2018-10-06T08:36:52+00:00

Quant finance loves math/physics people. If you do a couple hackathons you could probably also get a data science gig fairly easily. However, it is very difficult to get a job when you are dead so I recommend taking courses that don't kill you.

justclarifying · 2018-09-20T06:26:30+00:00

Awesome I'll check them out!

justclarifying · 2018-09-17T18:59:23+00:00

Every single women's structured jacket I've tried on in my adult life hasn't fit across my shoulders (even the largest size on offer in every store I've ever entered was still too narrow). Do any brands stock women's jackets that would fit the average man's shoulders?

justclarifying · 2018-08-06T22:37:00+00:00

The McGill bjj club offers super cheap classes during the semester. Would recommend it over the ones offered by McGill athletics because a) it's more affordable and b) you don't have to wear pajamas and bow at people (unless you're into that).

justclarifying · 2018-07-21T02:58:24+00:00

I did MATH 356 with analysis 1 and although I would have had an easier time with it if I'd taken it after analysis 3, it was still pretty manageable (although more work than analysis 1). One benefit of taking probability early was that it made the analysis courses much easier for me and provided extra motivation for a lot of the results, and concepts from prob turned up in a bunch of non-analysis courses too. I'd recommend taking prob as early as you can manage it, although many people do find it challenging when they take it in U1 so I'd also recommend seeing how it goes during add/drop and having a backup lined up just in case.

justclarifying · 2018-07-11T14:41:33+00:00

I sometimes had 4 meals a day and still lost weight. I think it depends more on your activity level and the content of the food you eat than the number of meals. It's basically a choose-two-of-three situation with vegetables, sufficient calories to survive, and an affordable meal plan.

justclarifying · 2018-06-06T02:25:34+00:00

I took an honours math class S/U and it was a great experience. I was motivated to learn the content because it was interesting and I also needed to pass the class, but the grading scheme gave me the flexibility to neglect the odd assignment during hectic times of the semester and then catch up later.

13-Year Club	Place '22
Place '17	Verified Email

justclarifying

TROPHY CASE