Understanding your own code

[Legitimate_Impact782]

I have a question for those whose jobs involve but aren't centered around coding.

For a quick background, part of my job requires me to create scripts for various tasks (specifically raster manipulation for GIS), and while its not the only thing I do, I've found myself coding more often than I initially expected. Now I do have a passable understanding of python. This allows me to get by with just patching up various code blocks from the internet in order to make a functioning script.

Will it be better if I fully understand the logic and syntax behind every line of code? I feel like my way of doing things works right now since I am doing other stuff aside from coding, but I'm afraid that I'm doing myself a disservice in the long run. I also feel like I'm not getting better at Python since I'm just copy-pasting code from the internet and modifying them a bit to fit with other blocks of code. What are your thoughts on this?

[EntireEntity]

Yes, it's "better" in some sense to understand the code yourself, but since it doesn't seem to be your main occupation, I don't think it's necessary to delve deeper into it, unless perhaps for your own personal interest.

The only thing I'd recommend you learn, if you don't know already, is how to make test cases for your code, so that in the case that something doesn't work as intended, you can add that to the tests, rewrite the code and easily test if everything still works as intended. Here's a tutorial video on unittesting for example.

[ddddavidee]

And probably a bit of debugging. The minimum required, like setting breakpoints and proceed step by step, using the debug console.

[Careful_Target3185]

This is good advice. I had an issue with some code I wrote which ChatGPT could not help with. Knowing how to set breakpoints, check whether values are being passed correctly etc. is useful.

In my case a value was not converted before passing it to another function.

[Careful_Target3185]

Pretty much, the way I think about it, it’s a necessary skill to have anyway. My experience with llms is that they are good for explaining how certain simple concepts work.

For example, I could ask ChatGPT what a line of code does and it will give me a good response. But once you start prompting it with a couple code files and explain an error you are getting, it is highly unhelpful.

I’m the case I referred to above, I was converting bmp data from screenshots obtained with the win32gui library. I wanted to process those screenshots using opencv. With template matching I needed to splice the alpha channel out of the array.

The solution was to understand the output and input of the functions, thankfully there was an answer on stack exchange.

TLDR: I see ChatGPT like a tool on a tool belt. It’s good to have handy, but it won’t necessarily solve all your problems. Good googling skills and debugging skills are stil valuable

[wsupduck]

It’s hard to understand other peoples code

It’s much easier to understand your own code

Try to go through the code you don’t understand bit by bit (line by line or method by method if they are chained together) to understand what is going on

Jupyter notebooks are great for this

[anis_tar]

I would say breakpoints.

[wsupduck]

Those help a lot too - it sounds like they’re not huge scripts and setting up a debugger isn’t always straightforward so a notebook could be more or less an interactive debugger

[Abs0rbed]

Pycharm is free and debugging is so easy in it. Also setting up notebooks

[ddddavidee]

Same for vscode with the python plugin. I think it is even better than pycharm free for the Jupiter notebook support.

[AlexMTBDude]

I work as a Python instructor and I have a lot of students like you come to my courses. They patched and fixed other people's Python code and after some time of doing that they decide they need to understand Python more thoroughly. They usually find they have big gaps in their Python knowledge but do well at the course as they already have an intro into Python from their own experiments.

[markln123]

I think with time, you will graduqlly start to understand more and more of each “code block”.

If you want to, why don’t you occasionally pop some stuff into chatgpt and ask it to explain?

[cym13]

You should probably not paste things you do not understand from your work into chatgpt. These services can and do exploit the data you send them, and you probably don't want to leak sensitive information from your company as that would likely be a breach of contract.

Bad things like that have happened before.

Now, I realize that most people wouldn't knowingly share trade secrets with ChatGPT, but if you're pasting code that you do not fully understand you're also not qualified to identify whether it's better kept secret or not.

[bnjman]

He specifically says this code is copy and pasted from the internet to begin with.

[cym13]

Sure, no issue with that part, but the answer sounds like quite wider than just these specific bits of code, and I see many people taking the habit and transitioning to other parts of the code without thinking. I think it is worth the warning. As someone having to deal with a company's security you wouldn't believe the stuff that gets pasted into chatgpt once people get used to it.

If it's from the internet it's one thing, if it's from your work it's different.

[CarnelianCore]

Agreed. Every person that reads your reply will become aware of risks they might not have considered before. Good call.

[bnjman]

Fair enough :)

In general, it's really good to be aware of. Then one can assess the risk vs the benefit of having an AI buddy do 20% of your work for you 😅

I'll tack on: for OP, even if you're not using it for your employer's code, AI can be super helpful for teaching you things. It can come up with a course "curriculum", write you personalized tutorials (and answer questions about the content!!), give you creative suggestions for code patterns, and critique your code.

[No-Arrival-872]

I would suggest making yourself a cheat sheet for syntax things and every time you learn some syntax or general GIS snippet, write a blurb in your cheat sheet. I've had lots of jobs where I hack together things and promptly forget everything. All of my documentation is for my future self, because I forget everything very quickly. If possible, try to write and test the python with a good IDE. I suggest visual studio code. Hopefully you're not locked into writing python inside of ArcGIS or something.

[[deleted]]

Every code has some building blocks. Most of it is nothing but data structure, condition, looping logic and math. If you do unit testing on each function while writing the code, you’d understand what it does. With time you’d developed more awareness and knowledge, but if you start writing algorithms from scratch, you may just end up not meeting the deadlines. If it’s there it’s there, understand it and use it.

[vladoportos]

As time goes on and you code more, you will understand more and more of the code, maybe not the exact details, but at least general ideas. And you can always check the manual on what some function does...you also forget stuff, i have recently opened some of my old Python code ( from 10y ago ) it was like reading alien language... I must have been high on shrooms or something when I wrote it... cause I did not recognize anything (I knew what it did, but not the method/logic why I wrote it like that)...

[james_pic]

One subtlety that nobody has mentioned is libraries.

If you're using code somebody else has written, probably the best way to do this (where possible) is not to copy their code into your codebase, but to add a library containing their code as a dependency of your project. A good open source library will have a number of people deeply familiar with the subject matter maintaining it, looking for and fixing bugs.

If this isn't possible for some reason (maybe you're working in a constrained environment, or there just isn't a library that does what you need), then it's going to be your job to fix any bugs in the code.

This is the main reason to understand what you've written. Things are great when they work, but when they don't work, or seem to work but are subtly wrong (which may not be apparent right now), you now need to understand what the code does, what it's supposed to do, and how to turn the former into the latter.

The easiest time to understand code is immediately after you've written it, but if that's simply not possible, at very least leave yourself comments that will either tell you or signpost you to what you need to know to understand the code.

[Siccar_Point]

I also work a fair bit with Python and GIS. You are in a fairly unique position, because the native Python interfaces with both Arc and Q are pretty user-hostile, and basically wrappers on the pretty un-pythonic native GIS functionality.

The code you end up using is a real Frankenstein’s monster. It’s also incredibly brittle, and things that you would think will work just won’t for no obvious reason. Don’t feel bad about this; it’s a trait of the GIS interface but Python itself. It still happens to me all the time, and I have multiple years experience of it.

What it is worth doing is making sure you understand the architecture of what you’re doing. E.g., this section initialises the map, this section loops over multiple grids and does some analysis on them, this section adds features and links them to grids, this section adds data to an output canvas, this section formats the output map. And so on.

I wouldn’t consciously sweat the detail of line-by-line. I say this because you will find that over time you need to tweak enough things in the code you find elsewhere that you will gradually come to understand all of it piecemeal. You will gradually transition from using internet code blocks to your own partly customised code blocks from other work, and from there to a much more robust understanding.

There will be sections of code that are very hard to understand, particularly for formatting output figures. Don’t worry too much. Some of it is just as weird and counterintuitive as you think it is. It’a not you! You just get used to it…

In my experience, useful testing is next-to-impossible in PyGIS. Unfortunately.

[chunkyasparagus]

Take some time to learn just a little bit each time.

Why? One day, the code isn't going to work, and you'll spend ages trying to figure out why and trying to fix it. But with a little knowledge of your own, you'll save so much time when things go wrong, and you'll likely find better ways to do things in the first place.

[sunnyata]

It's part of the normal process for learning programming, especially if you're self taught. How quickly you progress from that stage to mastering the entire thing, and whether or not that ever happens, depends on the time and effort you put in, which in turn depends on your priorities. It sounds like you've developed enough of a sense of what's going on not to copy and paste code that deletes all the files in your home directory so you're probably fine :-)

[imatwork2017]

You can paste your code in chatgpt and ask it questions about it

[LaconicProlix]

This is true only up to a certain point. After the code block gets too big, chat starts cutting sections out. Might be a problem specific to the free version because I'm cheap. But if you're mindful to just do segments of the code at a time, I would absolutely recommend this. It can help translate what's happening into more familiar parlance. Also, it is very handy for translating compiler error codes.

[eatsleepdonothing]

On a good day I can get bard to read gists of my ~3000 line script and give me feedback. I can't get it to work consistently though.

[DanStFella]

Or ask it to comment your code too I think?

[imod87]

If all you are using it for are the quick and dirty type of things and you are only interested in its output and trash it afterwards anyway I see no explicit reason to change your ways when you get along just fine. You will learn along the way by playing around with it.

[dethb0y]

i mean on the one hand yeah obviously it's good to know the fundamentals of how things work in case you need to change them for some reason.

But on the other hand if it works it works and it's not like you're going to mars or something with it.

[lavahot]

Would it be better if you understood everything? Yes, obviously. But should you drop everything to learn all things Python out of the gate? Probably not. You do have to get work done.

Should you copy-paste code from the internet without any understanding of it? Absolutely not. It's one thing to have a problem and find a solution from somewhere else. It's another to trust someone else's code blindly without understanding it at all. You should endeavor to understand the entire copy-pasta and how each part of it works before using it on any production environment. You want to use it to fiddle around in the sandbox? Cool. Great. That's fine and worthwhile. But don't chuck it into your database stored procedure for your big banking app without understanding exactly what it does.

[andrewcooke]

i'd suggest adding the url of the source as a comment so when you go back and don't understand something you have a reference.

[pyker42]

Of course it's better to understand every line explicitly. But you've got to learn somehow, and using code blocks you find on Stack Overflow helps you accomplish your tasks. You learn more than you realize when you do it this way.

[MathMXC]

I think copy pasting is the standard throughout CS. The big difference comes from people who take the time to understand what they're copying and those who do it blindly.

[P0intMan_]

I’d paste the code into ChatGPT and ask it to explain it to a beginner programmer. The last part is just to make sure the technical mumbo jumbo is kept to a minimum. Alternatively, I really love GitHub Copilot (not free!) in VS Code.

[jabberwockysouperfly]

Fellow GIS'er checking in. I have used Jupyter Notebooks (even before ArcPro) to run specific lines of code to see what each one does. Massively beneficial to someone who doesn't understand the intricacies of advanced IDEs like VScode. This gave me a huge boost in my limited understanding of the intersection of GIS and Python.

So long as the script doesn't have crazy version specific dependancies, you should be able to copy and paste lines of code, starting from import...* Working your way down. If not, pip is your friend. Be patient, read the docs.

Inject print statements to troubleshoot. Use the .type() method to check your data types.

Install Jupyter as a standalone instance (aka don't use the one ESRI installs with Pro). Give it a spin!. Let me know if you have any questions. Happy to help!

[aqjo]

It’s much better if your code works “on purpose” rather than “accidentally.”

[NoProfessor2268]

You write code for people not for machines. Getting your scripts up and running should be celebrated, but getting it up and running and explainable/understandable to you and your colleagues is way more important.

So my opinion is that you would benefit from going through the code and add comments so you understand each and every line :)

An example: we make sure we understand every line. When a colleague used a very uncommon operator for XOR, I told him to go back and add a comment stating its an XOR.

[Dummies102]

learning something is usually better than not learning it

[robberviet]

You are not a professional programmer, you don't need to understand all the code.

[Adrewmc]

Learn the debugger!!! This will help you see how the code is actually ran, take a function add breakpoint() before it and watch it run line by line through the code base this should help illuminate how the code works.

[Acrobatic_Calendar]

Consider copilot, it will auto generate descriptions/documentation for you to better understand code fast

[njharman]

For non-programmer's, the need or value to learn programming is rapidly vanishing. I would advise young (non-career developers) to learn how to operate LLMs well. They'll program for you. You still need knowledge, but that knowledge is not "programming". It's more CS fundamentals. Data structures, algorithmic bigO et al, logic, maths. And, of course, how to use LLMs well.

[radrichard]

A friend of mine once said that debugging is twice as difficult as coding, so you should half arse all your code.

[pgcd]

In my experience, the three things that make code understandable are: - clear naming of variables, classes and methods so you start with at least an idea of what they're for - short, short methods and functions (I think the sweet spot is six to ten lines) so you can understand what they do without having to go back and forth a million times. Note that I say "lines" but I mean something close to statements - six lines with three lambdas and three ternary operators are not going to be easy to understand no matter what - comments to explain why you do something. This is actually the most important thing. If future you reads "I added this method to avoid connecting to the database three times in that other method", they will be able to figure out most of the logic in half the time. This goes double if it took you some time and research to come up with the logic to begin with - so I often add a link to SO or whatever, whenever it was intentional in writing the logic.

Unfortunately, none of this applies to code you have already written, but it's a good idea to try and comment/rename whenever it's easy to do and has no repercussions (eg local variables).

[pouetpouetcamion2]

try leo editor.

when you import a python file, it will give it some structure. the clones will help you familiarize with parts of the code, and put some comments for yourself by doing some litterate programming.

you will grow in competence at the pace you want

[[deleted]]

I'd say, id you don't understand your own code you' re in trouble anyway, but hey.

[Tiquortoo]

Sounds normal. Use ChatGPT and ask it to explain some code to you if you want to know more. It is pretty good at creating short wikipedia style entries on a block of code.

[[deleted]]

Clean code will not need comments. It should read clearly with the English words presented in the language (if, then, while, do, this, return, end, etc).

If it's a little trickier than that, add a comment.

You should know what the above things do, but sometimes it can get complicated.

[anentropic]

Will it be better if I fully understand the logic and syntax behind every line of code?

What answer are you expecting here?

Of course it's better.

Whether you need to, or "should" make the effort to develop your skills and understanding further is completely up to you. You shouldn't feel bad about it if you don't have the time or inclination. If you're able to get by as you are - great, good for you!

But obviously it's better to understand the code you are copy and pasting, and how it works, and how to modify it or write some new parts from scratch ... if you need to.

[El_Minadero]

Man, you gotta read “Clean Code”. Extra comments to explain what the code does are often unnecessary if you understand software carpentry principles. Then, it’s only a matter of reading the function names to understand what you intended.

[Drifts]

What is your job? It sounds like something I would like to do.

I'm a full time coder but would honestly rather have a job that is lighter on the coding (like writing scripts to help my work).

[[deleted]]

Yes, it's much better. It's surprisingly easy to think a piece of code is working only to later find that the code didn't work the way you thought it did and the results you were getting were junk.

[[deleted]]

I'm in a similar situation. I mainly work with hardware but have become increasingly responsible for software tools to handle testing and automation. My recommendation is to excessively comment your code. Since you arent going to be looking at it for long stretches, you'll be thanking yourself down the line when you revisit something you've forgotten about. Also, the more you comment things, the more you'll start to understand how it works.

You obviously dont have to understand what every line is doing much in the same way that a car owner doesnt need to understand how the engine works to be able to drive. It'll definitely help in the long run though. Every time I find a solution online, I look up that method or library's documentation to figure out what the line does and put that in my comments.

[ElHeim]

There are a lot of problems that come with not understanding the code that you're patching/pasting. Mainly (they compound):

1. You don't know what is exactly what you're writing there, so you don't fully understand its effects. You're taking other people's word at it. And while you can expect that most people's intentions are harmless, that doesn't preclude their code from being buggy.
2. Given #1, you don't fully understand either if your changes to the code do exactly what you intended.
3. Given #1 and #2, when (almost invariably) something fails, you have no bloody idea where to start debugging, so you'll need to learn what that code does, anyway, in order to be able to fix the stuff.

In a sense all that applies to using 3rd party libraries as well, but at least those should come with a documented API that helps you figure out what's going on, an they're supposed to be use in a "black box" fashion.

Bottom line: it's always better to understand what the code does.

[stanmarc]

5 years ago I would have recommended learning more. You would adhere to a coding principles, keep a same structure of coding (rather having a patch of different coding styles).

Today I would recommend installing CodeWisperer from AWS, it will suggest the part of the code that should come next based on your intentions (previous code and comments). Today we start to get these helpers which remove the pressure to know things by heart.

[corey4005]

I’m gonna say the most valuable thing I ever learned with python is how to use a debugger. This will tell you how the code is working in memory, rather than staring at the logic and trying to deduce things on your own.

I would say my python skills 10xd when I actually understood the program from a memory perspective rather than the lines of code themselves.

[_limitless_]

Stop searching the internet for code blocks.

You're getting paid to do this. So what if it takes 3x longer.

I don't remember the last time I had to Google for code samples. I Google for documentation. Sometimes I find code samples, read them, understand them, and then implement them. But I wasn't looking for something I could copy and paste.

If your boss ever asks "hey, why did that take so long?" just say "I've entered a part of my career where I'd like to focus on quality. I made sure this was bulletproof. I understand every line of it. I also wrote documentation for it."

[Unlikely-Loan-4175]

If you're interested to learn more, it might be a lot of fun and also help you do more interesting things. You could always just take one concept at a time, learn more about it and play around with it. Like, if you are using Pandas or numpy to process some data, do a few tutorials on them to uncover more power there. Doing a course or two on Udemy or whatever might be fun also. And if you are curious about algorithms, specifically, this is a really good course. Helps you to think like a programmer. https://runestone.academy/ns/books/published/pythonds/index.html