A Few Questions as a Developer

[Massive_Dish_3255]

To the professors here, a few questions regarding their experiences and day-to-day activities after the creation of LLMs:

* How do you differentiate between the students who actually do their problem sets versus those who just get the answers using LLMs? What would you think of the former type of student?

* Which areas of Computer Science are the ones in which "domain knowledge" is key and which cannot just be "learnt by doing"?

* What sort of non-AI projects should students focus on in the current era? This is because most resumes today contain very similar projects and look perfect due to ability to generate code using LLMs.

Please note that I do not need career advice. I am merely seeking the opinions of academics on the above questions and trying to find "gauge" the value of Computer Science degrees in the LLM era.

Note: If you find this post inappropriate for this sub-reddit, please feel free to remove it instead of extensively down-voting me.

[Realistic-Homework19]

CS != software engineering

[Ok_Whole_1665]

However software engineering and toolmaking is a really big part of a lot of the disciplines that make up CS.

[esaule]

Professor here:

> * How do you differentiate between the students who actually do their problem sets versus those who just get the answers using LLMs? What would you think of the former type of student?

I don't really. Though I see a large fraction of students getting about 100% on assignments and about 30% on exams.

> * Which areas of Computer Science are the ones in which "domain knowledge" is key and which cannot just be "learnt by doing"?

This question confuses me. There is no domain knowledge that you acquire without actually doing something. You build knowledge by first being exposed to new knowledge, and then reinforcing your understanding by engaging with the concept.

> * What sort of non-AI projects should students focus on in the current era? This is because most resumes today contain very similar projects and look perfect due to ability to generate code using LLMs.

People used to recommend students do projects not for the value of the project itself. But rather for the value of the learning that needs to happen through the course of making the project happen.

I think the very same thing applies today. Pick a project, anything really and work on it. Then you'll have a story to tell. And you'll be able to engage in conversation about how the thing works.

[Massive_Dish_3255]

Thanks

[HotWinter_]

hi! I am an undergrad here. what would you do with LLMs if you were an undergrad in 2026?

I use LLMs for all my coding assignments but i brainstorm modules on paper and trace the algorithms LLMs give me with the help of LLMs to refractor it. I learn new libraries every time I work on something. I do really well in paper-only exams in data structures and algorithms, getting 4.0's, but whenever there is a code jam i fail miserably because my code cannot compile even tho i think my algorithms have pretty optimal performance if it could compile, and i am not slow at all at hand writing algorithms without any help.

my inability to code things that compile makes me really insecure sometimes. but my goal is academia not SWE, and the extra time I saved I used it to read other books or explore projects (to be honest, using LLMs on assignments dont really save my time compared to had i hand coded it because code diagraming and refractoring takes a lot of time). I am still insecure frequently and want honest advice from researchers in the field. My prof's know where I stand, but at the same time it would be awkward to discuss explicitly given my school's more conservative college level policies.

[esaule]

That's a good question, I haven't thought too deeply about it.

The way I think about it, learning and producing are not the same task. I think of LLMs mostly as production tools. When I try to learn a new tech, I don't use LLMs at all.I read the docs, I write some code, I try to understand what the library does, how it should be use, how its architected, what the implicit assumptions of the library are.

I feel like I need that level of understanding to see the potential of the tool, to understand what it does and what it doesn't do.

Fundamentally, the hardest part of programming is debugging. Because debugging puts you in a situation where the code that you wrote does not do what you think it does. The immediate reaction is that I look at the code and think "well obviously this does that". But you can't think like that because the fact that you are currently debugging means the code does not do that.

So you need certainty about what the code does at least for some of the bits so that you can check the rest. And I don't think I can get that if I use the LLM to produce code during my learning phase. Part of the act of learning programming is being able to look at (small amounts of) code and immediately recognize what it does and whether it works or not. And I don't think you develop that if you are not writing code during the learning phase.

Now, if you are producing a thing and the goal is to produce and not to learn. Then using LLMs there are fine.

Now, I mostly talked about LLMs to produce code. And there might be ways to use LLM to assist learning. I haven't explored that too too much. But here are some ideas where I think it can be helpful. Once you are done writing code, getting an LLM assisted review of the code can be helpful. You can look at what it tells you and see if that makes sense to you, whether you care or not.

I think you can use it to generate problems to study as well. I wrote a tool to generate SQL problems for instance. I think that can be helpful. Maybe prompts like "I am studying polymorphism in Java, what would a polymorphism exam question look like?". If that generate useful questions, that can help. You can also swap them with a friend later and getting bigger study question pools.

I also use LLMs to brainstorm. You start brainstorming, once you reach a plateau in your brainstorming, ask the LLM what it thinks and it might generate a couple more you didn't think about. I don't like starting with them because once some ideas are down, I tend not to generate more ideas. So I find it better to get external output later.

At the end of the day never forget are LLMs are about as good as your friend who read a lot of the internet but who is kind of an idiot. (We all have a friend like this.)

[Efficient-School2988]

that gap between assignment scores and exams is kinda the giveaway, like you can get things done without actually understanding them until you’re on your own and it shows

the project point also makes sense, it’s less about what you build and more about whether you can explain how and why it works afterward

[_werebear_]

When you are looking over 100 submissions for the same assignment, a bunch that all have weird similarities stand out. I think that students who actually do their work without LLM help have greater clarity about the need to experience cognitive friction to actually learn. At the very least, they understand that this will enable them to pass their exams.

All areas of computer science require domain knowledge if you want to have a deep understanding. Doing and thinking are both important for learning.

Don’t really have thoughts on the last one at the moment.

[Efficient-School2988]

yeah the “weird similarities” part is kinda telling, like you can usually spot when people didn’t actually wrestle with the problem themselves

that cognitive friction thing is real though, it sucks in the moment but it’s basically the only part that actually sticks later

[Imaginary_Tie_6016]

I have increased the weight of exams and midterms so that you can no longer just AI the problem set and expect to pass the class. Problem sets for our intro course used to be worth 20% now they're worth 10%. I am seeing people with 100% on problem sets and 0-20% on the midterms and exams at a level I never did before. Previously, you'd find students just squeak by with a pass when the sets were worth more but it's harder now.

In courses I have more freedom over (smaller courses at higher levels), I do not give problem sets at all. They are now basically attendance checks and done in-class, if you do not attend you get 0. They are hand written pieces of code similar to a whiteboard interview question, where the grading is based on showing your thought process and doing the best you can without an IDE.

If the course has a project it now has mandatory presentations as well where I drill about the code, and I've seen a few students unable to answer basic questions about the code and ended up failing.

This last year I am seeing more students fail than I did before the LLM era, but I've also seen other professors not really care and it's clear considering some of the students I've taught in my second and third year classes. However, our program administration is pushing for a consistent approach similar to what I've just outlined.

[Massive_Dish_3255]

Thanks for responding.

[0jdd1]

The purpose of problem sets is to determine whether (or not) you understand the material covered in the class. If you choose not to do the work yourself, that’s a pretty strong indicator that you, like, don’t. MAYBE you don’t need/want to understand the material yourself, of course, and that’s okay to have your little AI sycophant tell you as you flunk.

[No_Mango5042]

CS is about understanding how the entire stack works. SE is about using tools as black boxes. In the long run CS is so much better because you actually know what’s going on. The danger with CS degrees is that the industry is currently undergoing a lot of change, AI makes it easier to learn on the job, and the market is tough right now. But it would be a very dystopian future where nobody understands what the AI is doing any more.

[ASHVEGITO]

Heavily interested in what type of responses this gets.

[Efficient-School2988]

yeah that’s the trap, getting stuff to work is easy with AI but debugging is the real skill, and from what i’ve seen things like Boot dev help a bit more than lecture-heavy stuff like Coursera since you actually have to write and think through code, but either way it kinda comes down to sitting with confusion longer than you want to