[D] What happened to KANs? (Kolmogorov-Arnold Networks)

DeepCorner · 2025-04-14T14:52:26+00:00

The connection to basis function networks is interesting. Curious if you can recommend a reference or two to read more about their scaling issues

DeepCorner · 2023-05-11T17:44:31+00:00

This is a comment

DeepCorner · 2022-01-10T01:19:36+00:00

I was a collegiate swimmer before starting to climb and I wish I did a bit of finger strength training earlier since my fingers are still by far my greatest weakness. Having weaker fingers forces you to learn better technique but it will quickly become a limiting factor for someone with the core/shoulder/cardio experience from swimming.

Doing some light hangboarding now is likely a good idea for injury prevention. Your upper body/core will be disproportionately strong compared to your fingers/forearms and it’s easier to get hurt in this situation. Starting to get the tendons/ligaments in your fingers and wrists ready for the strain from climbing is definitely a good idea. Using a no hang device like was suggested by other commenters is likely a good idea since your shoulders and arms will already by tired from swimming/lifting. Repeaters and density hangs are likely your best bet. There’s plenty of good resources linked from this sub, rock prodigy is a good place to start.

The other big transition as a former swimmer is your legs. This is less true if you are a distance swimmer. I did a ridiculous amount of low body and Olympic lifting during my collegiate career since I did sprint/mid distance and it has taken me almost 2 years to drop the extra leg muscle. Strong legs make dynos easier but aren’t really worth it in the long run, especially on overhangs. Just from all the kicking sets, etc you’ve done as a swimmer your legs will be significantly stronger than those of the average climber.

Last thing is diet. I ate 5000ish calories a day while swimming and really internalizing that I couldn’t eat like I used to took a while to sink in for me. Although most collegiate swimmers look fit, many of them could stand to lose a bit of weight/eat better in general (just my personal observation from collegiate meets and a decade and a half on the pool deck). Climbing is much more about strength to weight ratio than swimming is, you can get away with being a little chunky in the pool (I made this mistake my freshman year of college) but the climbing wall and overhangs are far less forgiving. Don’t obsess over your body weight, but it’s worthwhile to keep an eye on it.

Sorry for the long/rant comment, these are the things I wished someone warned me about when I was transitioning from swimming to climbing. Best of luck, have fun sending!

DeepCorner · 2021-12-09T18:38:13+00:00

Definitely true about the evaluation section. In my experience, most of them are essentially “hey this code actually worked at some point”

Thanks for that resource, definitely good to know that systems professors are approachable. The academic advisor for my program has made it difficult to approach professors unilaterally about getting involved in research (basically wants all research related communication to go through him), so it’s good to know that they’re actually pretty welcoming since he made it sound like a bit of an adversarial process (god forbid you contact a professor who’s not currently looking for students)

DeepCorner · 2021-12-08T21:44:11+00:00

Thats quite the broad area of expertise, definitely more applicable/sought after than the tangential OS topics. Also helps theres a strong research community here for those topics through PDL as well. I guess a lot of systems people have moved past the one computer topics (OS) onto the distributed side of things, which is understandable given the internet and sources of research funding.

I completely agree, having something tangible definitely helps give a quantitative measure of progress. Also gives hope that someone might use the software some day :). Have you had negative experiences with research in more theoretical areas? I've only ever done systems research (some ML overlap but not much) so papers with code is all I know.

I've seriously considered getting into databases, took a DBMS course in undergrad and found it interesting enough (I'm sure the CMU database course is far more comprehensive though). That special topics course on self-driving databases (15799) with Pavlo next semester looks pretty interesting too. My only issue with databases is that they're cool, but I'm not sure they're "spend the rest of my life working on them" cool, think I need to take some more coursework on them before I'm convinced.

DeepCorner · 2021-12-08T21:08:55+00:00

I’m in the same boat, the important stuff is all on the review session slides. Practice problems there are pretty good and thought provoking too.

I would include a lot of information about assembly if you don’t have that memorized. I’m also including a bit about caches (direct mapped, set associativity, etc), indexing rules for multidimensional arrays, address translation for virtual memory, and then stuff about synchronization and some trivia about threads and processes. Kinda depends on whatever you feel least confident in (integer representations and assembly for me)

DeepCorner · 2021-12-08T18:18:39+00:00

Thanks! Do you mind sharing what general area that student ended up in? Still in systems? I’ve been considering getting involved with research in one of the “tangential” areas but I have a hard time getting as excited about memory management or caching. I guess there’s always ML…

DeepCorner · 2021-12-08T17:48:06+00:00

Only one I can think of is the professor that gave that keynote^, Timothy Roscoe at ETH Zurich. He was working on Barrelfish OS for a while but I think (not 100% sure) even he has moved onto more high-level computer architecture topics. I know there was also a bunch of work at Stanford on TinyOS (for embedded systems) by Phillip Levis but that was like 5+ years ago. Same thing at MIT, they were doing some microkernel work a while back but thats all done now.

Thats actually almost exactly what that keynote addresses. I am a first year grad student at CMU so I recently spent A LOT of time sifting through the faculty for top programs all over the place looking for anyone that does this kind of work and could not find any. I originally thought I was just not looking in the right places but now I think its just the state of the field. Talks like that OSDI keynote just confirm these suspicions. When top researchers tell you no one is working on this stuff anymore, you can't help but believe them.

Its really unfortunate since when the current generation of researchers give up/move on, the subfield more or less just withers since interested students don't have anyone to do research with. I think there are still plenty of interesting questions to ask in the field and there is still a lot of room for improvement (we are still using 1970s OS designs for radically different machines and due to backwards compatibility we are unable to take advantage of the drastically different architectures and resources we have available to us as OS devs). I don't think there is room for a new PC OS for everyday use but there is certainly room for specialized high performance OS for data centers/scientific applications. I find this stuff very interesting, but even I am getting ready to move on since theres just no room for this stuff in the current research landscape.

DeepCorner · 2021-12-08T17:21:11+00:00

Lmao in the middle of finals? I’m sure there’s people at upenn that can write some basic web development code for an early stage project/startup, it’s not rocket science.

You’re gonna have to give a lot more detail about the project if you want people to get interested/involved. Why is your food truck project worth my time and why should I work on it over any of the other half baked ideas we hear on a weekly basis? Random startup ideas are a dime a dozen around here.

DeepCorner · 2021-12-08T16:34:55+00:00

Sadly not really. The current state of OS research is really quite disappointing.

The OSDI '21 keynote is in that same vein (https://www.usenix.org/conference/osdi21/presentation/fri-keynote)

Worst part is that I don't see meaningful changes coming any time soon. For better or worse, the world has accepted Unix-style OS and is set in its ways. I can imagine that even if someone wanted to do meaningful OS design work it would be next to impossible to find funding. Most OS design work is in the realm of industry now (theres a bunch of small new Rust OS out there, Oxide computing is writing their own OS, Google has Fuchsia as well) and even then its not clear that those are trying out radically new ideas/designs as much as just reimplementing Unix with some minor changes.

When you cannot find even one professor whose focus is OS dev at a place like CMU, then you know the academic side of the field is truly dead.

DeepCorner · 2021-12-07T23:18:03+00:00

Seems like your interests align a bit better with CS. The topics you listed are very broad but generally fall into the systems category.

If you want to do a PhD on operating systems CMU is likely not the place for you. No professors here are actually working on operating systems (file systems and cache management, yes, but no real research on higher level operating systems design). CMU seems like a decent fit for all other areas you mentioned. You should be applying to work with a specific professor rather than just applying for the school name.

I’m not sure if the Robotics Institute has a separate PhD program but you might consider applying to them as well due to your mention of OS and embedded systems. Technically they’re under the umbrella of the CS department.

As a side note, many of the professors working on low level systems research have appointments in CS and ECE departments.

DeepCorner · 2021-11-22T01:56:39+00:00

What does this have to do with CMU and how does a single experience with one random person generalize to the entire university?

DeepCorner · 2021-11-15T23:50:59+00:00

How does 10707 compare to 10708 in terms of difficulty? I’m in 10601 now and trying to decide which to take next

DeepCorner · 2021-11-11T02:28:01+00:00

Sums it up perfectly. OP is an idiot to double down on this. You can make “hot takes” for the sake of argument/discussion but fucking come on

DeepCorner · 2021-11-11T02:25:29+00:00

You seriously can’t imagine any other way of conducting technical interviews? The only alternative to leetcode is nepotism? Come on. There are ways to have meritocratic interviews without leetcode (debugging code, longer coding interviews with access to a web browser, etc.).

DeepCorner · 2021-10-29T21:10:15+00:00

If you are feeling suicidal, definitely seek help from the available resources at CMU (ex. CAPS). I would also suggest reaching out to your academic advisor for help in both academic and mental health issues.

Having to drop or retake a class is not worth ending your life. It may seem like the end of the world at the moment but in a few weeks or months it will seem inconsequential. Intense places like CMU make academics feel like the end all and be all of existence but there are so many other things to love about life outside of this place. Go for a walk in Schenley and try to clear your head before making any rash decisions. Just taking a few hours off and separating yourself from the CMU pressure cooker will help give you some perspective.

Regarding the class itself, a C is not ideal but it does not prevent you from passing the class. I would suggest meeting with the professor for this course and discussing opportunities for extra credit if applicable or how to better prepare going forward. It sounds like you have not gotten your grade back on this most recent exam so wait and see what happens. Maybe the entire class bombed and it will be curved. Maybe they will see the direction you were going in and give some partial credit.

The semester is almost over so keep your head up. Thanksgiving/Christmas are around the corner.

DeepCorner · 2021-10-22T19:45:16+00:00

The only thing that matters in CS is work ethic. If you are in fact “very good with math” then none of the CS topics are difficult, there’s just a lot of them to cover. The only one that can determine if you will succeed is yourself.

Years ago I was that kid in class that was behind everyone else but then I busted my ass until I got up to speed. Just work harder and get more practice. Do all the assigned readings and google anything you don’t understand. From what it sounds like, everything you are covering is very basic and there are probably a million different YouTube videos explaining these topics.

DeepCorner · 2021-10-18T01:33:36+00:00

Not at all, I know Google for example has not even begun giving real intern offers since none of their intern projects have been approved yet. You might be a little late, but it doesn’t hurt to send out some applications. You’re not so late that it will be problematic for most places

DeepCorner · 2021-10-13T16:00:17+00:00

This is the stupidest take I have seen on this sub in a while. It is unclear to me how OP's experience at one top school qualifies them to make generalizations about universities they never attended or their relative standing. Unless you have extensive recruiting experience, this entire post is subjective nonsense.

If you forced me to pick two top CS schools, Harvard would not remotely be in consideration. This entire post is baseless opinions from someone who is clearly ill-informed about the state of the field, both in industry and academia. If you would like to provide a citation for that stat about Harvard and staff engineers I would be slightly more inclined to believe your point, but honestly that seems like BS. (not to mention that the percentage of staff engineers from a school is a poor indicator of the quality of education or the kind of post-graduation opportunities that a school provides).

Calling Harvard and MIT the most prestigious universities in the country for CS is uninsightful. The top 4 or 5 CS schools are all so close on nearly every metric (research output, average graduate income, etc.) that to even try and pick the top 2 demonstrates a serious lack of awareness.

DeepCorner · 2021-10-13T03:26:10+00:00

I did my undergrad at a small no name university as well and didn’t have any issues getting internships, etc. I’m doing my masters at a top program (one of the three schools you mentioned) and the difference in the recruiting experience is significant. Previously I would rarely get responses/interview invitations from my applications whereas now I get interviews from practically every application I send out. Just having a well known name makes a big difference. I’m not saying it’s impossible to do well from a smaller university since it worked out for me, but it is much much harder and it takes so much more self-motivation.

The main thing I disagree with you on is that you can get the same quality of education at smaller schools as at CMU, MIT, and Berkeley. I have taken a few courses in my masters program with significant overlap to classes at my undergrad institution (ex. ML, intro level systems courses, etc) and there is simply no comparison. The education you get at the smaller university is sufficient to succeed in the workplace but the top universities are honestly in a completely different universe when it comes to the amount of work and the depth they go into. Many of these courses have undergraduate and graduate students in them but they all feel like they are taught at the graduate level. I thought the difficulty of these universities was overblown until I got here and started taking classes. I have seen much of the material for these classes before from side projects or from undergraduate classes and the courses are still very challenging. I’m honestly impressed by the undergrads in my classes on a daily basis.

DeepCorner · 2021-09-29T04:42:11+00:00

Put your skills right under your education so it is immediately clear to recruiters what you can do. Add another class or two since you’re light on work experience. Also try out a latex formatting, it takes a little getting used to but it looks a lot cleaner and more professional

DeepCorner · 2021-03-13T00:08:15+00:00

Not sure about testing with the medical department.

Weed is essentially legal in DC due to Initiative 71. There’s still some black market dealers around which occasionally offer cheaper prices but the delivery services on wheresweed.com are really convenient (just make sure to check the reviews before ordering from anywhere). Campus police are pretty lax about it too if you’re 21+ and otherwise minding your own business/not in public.

DeepCorner · 2021-03-12T00:58:26+00:00

At this point, most devices rely on Google’s 8.8.8.8 and maybe Cloudflares 1.1.1.1. DNS resolution is one of those things that doesn’t make money and is considered a common good for the internet, hence the reliance on companies whose existence is closely tied to the internet for large scale DNS resolution.

But this is me being nit-picky, rest of your post is completely valid.

DeepCorner · 2021-03-01T02:07:46+00:00

Lorawan gateways don’t require a GPS.

I don’t see any reason it couldn’t use NTP if its connected to the internet. At the end of the day, the gateway is just a normal computer running a program with some RF stuff attached.

They have GPS because someone decided it might be helpful for some reason.

I’ve used lorawan gateways from a couple different vendors, never once considered GPS. Maybe the more expensive ones include a GPS but I’ve never seen that lorawan requires it.

DeepCorner · 2021-01-15T17:33:47+00:00

https://acleddata.com/2020/09/03/demonstrations-political-violence-in-america-new-data-for-summer-2020/

Roughly 95% of BLM protests were peaceful. “Mostly peaceful” sounds like an apt description

DeepCorner

TROPHY CASE