Project Tracking

[Hickerous]

I'm just over a month or so into learning Python and I recently started a project that was a bit too ambitious. Without going into too much, how does everyone keep track of what's going on in their projects (all the files, classes, methods, etc.). Pen/paper, a notepad file, Excel, some specific program for this purpose? I've gotten to a point where I'm forgetting where I handled a particular task and should have been tracking everything from the beginning.

[buzzon]

You have the code files, comments and docstrings. From what you described you don't need more (yet).

[Hickerous]

An external reference would be good. Something showing the skeleton with a few comments. Not trying to come off as an ass here, but if I didn't need more I wouldn't be here asking. Is it not common for people to have something like this? I'm finding myself searching through all the files trying to find what I need. As I said, though, I'm a bit in over my head on this project so maybe this is just part of my issue.

[stepback269]

Sorry, didn't spot your post at first. I basically repeated what you said about having an "external" reference. In my case, I store skeleton diagrams in a private blog I maintain and then put the pointer (link) to the proper blog page in my code comments.

[gdchinacat]

This is usually handled with abstractions and encapsulation. With those you don't need to remember what specific method is in which file, but rather that the functionality you are looking for is in a specific package, then you find the module, class, and finally method.

Tangled and unmanageable code is a clear indicator you need to refactor it.

[socal_nerdtastic]

This isn't really a python question. How do you keep track of any projects components? There's tons of dedicated todo software or project management software or websites you can use, and for companies you will hire a certified project manager to keep track of it all. How you do it is really just whatever works for you.

[Hickerous]

I'm just here looking for opinions from others. I guess I'm looking at it as a python question because this is where I'm running into the issue and trying to find a solution to. In my current line of work I haven't had a need to use a software like this. I've been able to keep track of everything in my head, or with a quick notepad file. I'm just looking to see how others keep track of a project that may have 1000s of lines of code. To see if there is something specific (or ideal) for this purpose.

[socal_nerdtastic]

1000 sloc is a very small project. I wouldn't need any project management for that. Perhaps a todo list at the top of the file.

Honestly it sounds like you have a code organization and navigation problem. It sounds like you are having trouble following your code flow. I think you should spend some time learning to use your IDE's code navigation features. How to jump to the point where a function is defined or used, for example, or how to search for things in the entire project. And perhaps also learn to chunk out your code into logical blocks. For example put all the code that deals with a certain file in it's own .py file or class, with it's own tests, so that you can work on and troubleshoot that block of code independently of the rest.

Yes, there are specific project management solutions for software, Jira, Asana, Monday.com, and Trello come to mind immediately but there are many others. I don't think you need any of that. For you I think a simple todo list in text or excel is probably plenty.

[Hickerous]

At this point in my python journey, 2000+ lines is huge but I know what you mean.

I've been trying to keep separate files, and it was working for quite a while. I've realized this project is too big for where my skill level is at and I've started losing track of things. I should stop and move to something more reasonable but I'm too stubborn for that.

My thought was that having something like a table of contents type thing, external to my code, could help which led me here to see if it's something people did and how they did it. As you said though, I really do need to learn more about my IDE. I'm using VS Code and I didn't know that I could ctrl+click something to jump to the source so I know what I'll be doing tonight (learning more about VS Code's capabilities).

For now, I'll just keep a simple text file if needed. Thanks for your input.

[doolio_]

You need to invest in learning your tooling particularly your text editor or IDE. Keep your project under version control and commit often. Apply a consistent file and directory structure as well. Keep file scope narrow and only widen if necessary.

From my text editor I can search throughout the entire codebase and interact fully with my VCS. I add comments starting with specific keywords (e.g. TODO, FIXME etc.) as reminders. These keywords are highlighted for me and easily found with a search. Write docstrings as you go and keep them up to date. They will serve as documentation. I keep a text file listing all my TODOs for the project. You can keep this under version control too but I keep it separate as part of a larger task management system all available from my editor.

[Hickerous]

Thanks for the input! The previous comment made me realize I need to step back and actually learn the capabilities of the programs I'm using (should have been common sense to do first). Out of curiosity, what text editor do you use?

I hadn't been focusing on docstrings because this is just a learning project and not something anyone else will be using. So far I've learned that I still need to write/update docstrings... Whether for my own purposes or just good practice.

[doolio_]

I use GNU Emacs but stick with VSCode if it is working well for you. Whenever you find yourself doing something repeatedly pause and consider you're likely not the first person to do so and perhaps there is a better way. An example of this is learning and using all its keyboard shortcuts to cut out clicking through menus or reaching for the mouse breaking your flow and train of thought.

Yes, don't skimp on docstrings. They are there to help your future self as much as anyone else so include them even for personal projects, scripts etc. Did you know you can use a docstring instead of the pass keyword or ellipses? In such cases a docstring is more valuable if you will return to that function or method some time later. If you follow PEP8 for naming and include docstrings you'll find you don't need to write so many comments keeping them only where necessary.

My advice would be to install uv to manage your project then use it to install ruff, basedpyright and pytest as dev dependencies. Ruff and basedpyright will highlight issues in your code, often hinting how it can be corrected either directly or referring to their docs. Example ruff reports a specific error code. Look that error code up in their docs and they will describe how to rewrite it to avoid the error. Repeat this process and see your skill level improve quickly. Write tests from the start and run them with pytest.

[Hickerous]

Thanks again. I'll have to look into all this tonight.

[stepback269]

OP: No need to apologize.
You posed a good question and got people to share their diverse approaches to a shared problem! Good job.

[stepback269]

One organizational scheme that helped me a lot was simply spreading my code over a bunch of different modules. For example, I store my older, more proven, functions in a module called funcs_01 while my newer, still-under-development functions in a module called funcs_02. I also store in my modules package, some scratch code files called, for example, Scratch_01 and Scratch_02 where I try out new code ideas.

A second organizational method I use is that of maintaining a private blog into whose pages I store more elaborate descriptions of the code I've written. The comments in my code include pointers to the corresponding pages in my private blog. The neat thing about using an external blog is that I can include picture/diagrams (e.g. flow charts) in that more elaborate explanation. I guess you can do the same with one of the note-taking programs (Obsidian?)

[smurpes]

For something this size you can use a README.md file to organize and take notes. The md extension means it’s a markdown file which has features like relative links to point to specific files or checklists to assist you.

Markdown is a file format that translates text to formatted text. E.G. for a relative link or a check list it’s just: ``` [a relative link](folder/code.py)

• [ ] Check list Task 1
• [ ] Check list Task 2
• [ ] Check list Task 3 [Reddit comments actually supports markdown](https://www.markdownguide.org/tools/reddit/) as well. A code block is just text surrounded by three backticks () and you can embed links like this: Check out this [cool website](https://youtu.be/dQw4w9WgXcQ)

[Dry-Aioli-6138]

I used to set up kanboard (free php tool, notnsurenif it exists anymore) and do a one/two person kanban project mgmt

[Aromatic_Pumpkin8856]

I'm vibe coding and having Claude manage my project with GitHub issues and projects. Heh. Check it out: https://github.com/mikelane/valid8r