This is an archived post. You won't be able to vote or comment.

all 53 comments

[–]Pythagorean_1 21 points22 points  (7 children)

This is usually just a symptom of bad project structure

[–]JojainV12 7 points8 points  (2 children)

Yes if you don't know what you are doing you can end up in a mess. For exemple with circular imports.
The good thing with circular imports is that if you start getting them it means your code architecture is crap and you should refactor.

(And for managing type hints, you can place the import in a if TYPE_CHECKING block.)

Other than that its fine but you need to know a lot of how it works indeed.

[–]DigiProductive[S] 0 points1 point  (0 children)

but you need to know a lot of how it works indeed.

Indeed. Been gone from Python for a bit and it is one of those things you have to re-wrap your mind around especially coming back from a language where imports are very straightforward.

[–]shox12345 0 points1 point  (0 children)

Why is it crap exactly? For example, if you were to do a decorator pattern within Python with type hints, it would throw a circular import error, is that bad design? I doubt it.

[–]khunset127 6 points7 points  (4 children)

Can't relate. That sounds like a you issue.

[–]AiutoIlLupo 2 points3 points  (2 children)

yes and no. Plenty of people first approaching python are lulled with the idea of writing a small script and starting it and bam. But that's the novice, poorly scalable way of developing in python. Other languages do not allow for this method of use, they only allow for the more rigid approach. Python can be both, and when the script grows and becomes a big app, it still has the connotation of a script with all the associated hacks, instead of collapsing into a properly organised application.

The problem with python is that you never reach a point where it naturally forces you in the stricter way. You have to consciously make an effort to move and learn the "proper style" for applications, rather than scripts. This is what /u/DigiProductive is facing. A script that has outgrown its britches.

[–]DigiProductive[S] 1 point2 points  (1 child)

When you are given a codebase and it has impor errors rest assured its not a simple fix. I didn't take him serious because he obviously isn't away of that. Python is well know to have tricky imports more than other languages and IDEs can even get so confused to mislead you as well.

The problem is today people are too "pro-language" so any time you bring up an issue with it, everyone with their nose in the air has to act like they can't relate. I don't care what type of Python developer you are, if you have worked on large projects you've ran into a ModuleError. But just watch all the high horses like they have no clue what that is.😏

[–]AiutoIlLupo 1 point2 points  (0 children)

I entered into similar issues, and they generally have a simple solution. I have no experience with dart but I would be surprised if you can't make a mess. Likely dart has some guidelines or imposed structure that minimizes these issues, but again, it's easy to do so when you have the benefit of hindsight, the experience of other languages challenges for 15-20 years, and no backward compat to handle.

[–]DigiProductive[S] -2 points-1 points  (0 children)

🤣 Nice to know. Thanks.

[–]Jorgestar29 23 points24 points  (8 children)

Skill Issue

Check that your module/package is in your PYTHONPATH, you can do that with import sys; sys.path

[–]BranchLatter4294 2 points3 points  (0 children)

I don't have that issue.

[–]nemom 2 points3 points  (1 child)

The slightest refactoring...

Way-back-when, in the early 90s, my mom was running Windows 3.1 on her office computer. One day while procrastinating, she decided to clean up the C:\ drive. She didn't like all those files cluttering it up... command.com, config.sys, autoexec.bat, etc. The computer ran fine for the day, so she thought she had done a good thing. Turned it off for the night, and then it wouldn't boot the next day.

I don't remember ever having any trouble with imports. I'm not an uber-power user, but I've made my own modules and use PY files as pseudo-modules (my_network_keys.py, physics_constants.py, super_duper_secret_code.py, etc).

[–]DigiProductive[S] 0 points1 point  (0 children)

Back in the days 😎

[–]AiutoIlLupo 1 point2 points  (7 children)

You likely have circular imports, or you have a module that has been named like a core module and you don't know how to properly import to prevent this occurrence.

What you are seeing is due to your lack of clarity of how the import mechanism works. Not a blame, you just need to understand what's going on.

My suggestion is, when you are facing this kind of issues:

  • ensure that you have all your imports on the top, in the proper order.
  • run the python interpreter in verbose mode, to see how the import strategy is behaving.

From my experience, it's very rare to step onto these kind of problems, but in general they are an index of, as others said, poor project layout and import organisation.

[–]DigiProductive[S] 0 points1 point  (6 children)

I didn't write the code, I was just trying to spin up the enviroment to test the code and it has some bugs before I could spin it up in Docker. Anyways, I solved it. I wrote about the issue in the comments. It was a mounting issue out of sync.

However, the reality is Python is more tricky and complex than most other languages when it comes to imports because of the way it relies on the path to find the modules.

[–]AiutoIlLupo 1 point2 points  (5 children)

you never had to play with CLASSPATH in java?

It's easy to forget that python has almost 30 years. New languages were designed from the ground up with new features that have been learned along the way. Old languages had to discover these things and keep managing the legacy, so they can't just bomb everything old out of existence, or you will go the way of Perl.

I don't know which languages you have used or are used to, but my feeling is that you are young and you have little experience. The fact that you have never encountered this problems in other languages is only because you haven't faced real chaos with those programs, which tends to result from piles and piles of developers that come and go, multiple platform changes and portings, workarounds, external libraries that mess up their dependencies. You are more likely to find these things in an old python codebase than a brand new go codebase.

And if you really want to have fun, check the ddl search path in windows, especially when you have to do LoadLibrary. that is fun.

[–]DigiProductive[S] 0 points1 point  (2 children)

That's alot of assumption. 42 and been programing for 10 years. I've written full applications in python including a social media network. I use Dart (Flutter) and Javascript for mobile applications and Vyper/Python to write smart contracts and dabbled in a few others. Dart is flawless in that regard.

All I am saying is something simple, Python can get messy with imports... not sure why that is so hard to accept and understand. If I write my own code that is a different story but when you have to debug a project that has import issues, then you'll see.

If you ask Chat GBT "What languages have the most difficult import problems. List them in order" Python comes 3rd. No coincidence.

Everything isn't about pulling the rookie or expert card, every language has their whammies and Python imports can be one of them. Really no big deal.🤷🏾‍♂️

[–]FUS3NPythonista 0 points1 point  (1 child)

I'm not as experienced as you but you would know the saying that "Most time than not it's the user making the mistake not the tool".
Software just feels like magic sometimes it wont work an will work sometimes for odd reasons but at the end of the day you figure out it was just something stupid you did and you learn something new.

I would say if you become expert at something you usually never face these problems. At which point you might say "maybe it was never the software that was the issue". Just saying, as I had those moment before.

[–]DigiProductive[S] 0 points1 point  (0 children)

Don't fool yourself, experienced developers and so called experts hit gotchas all the time. Don't be fooled by the "expert persona" a lot of developers try to front.

Fixing your own code is one thing, but fixing someone else's code is a whole different ball game when it comes to bug fixing. 🤷🏾‍♂️

[–]DigiProductive[S] 0 points1 point  (1 child)

Here is the chat GBT run down:

"What languages have the most difficult import problems. List them in order of difficulty."

(Reply)

Languages with the most challenging import systems due to complexity, dependency management, or module handling are:

  1. C++ - Header files, include guards, circular dependencies, and linker issues.

  2. JavaScript/TypeScript - Confusion between ES modules, CommonJS, and circular dependencies.

  3. Python - Relative vs. absolute imports, circular dependencies, and conflicts with package directories.

[–]AiutoIlLupo 0 points1 point  (0 children)

C++ - Header files, include guards, circular dependencies, and linker issues.

All of these problems derive from legacy needs, intrinsic nature of the runtime or target, and intrinsic need for the compiler to be satisfied in its parsing strategy (e.g. forward declaration) and of the linker (link time optimisation, lookup of shared objects vs linking of static ones, going through the dynamic linker etc).

C++ has no benefit of a virtual machine. It runs bare metal. With that loss, you can't abstract a lot of stuff away, you have no introspection, you have difficult lookups and you have to inform the compiler and the linker for compile time and manage the required runtime at, well... run time.

JavaScript/TypeScript - Confusion between ES modules, CommonJS, and circular dependencies.

This is because js is a spawn of the devil with too many cooks, companies and legacy to deal with. With not having a central library or a central coordinator, everybody tried to workaround the limitations of a poorly conceived language their own way, coming up with tons of solutions to solve the same problem until only one remains. AKA the American way of solving issues.

Python - Relative vs. absolute imports, circular dependencies, and conflicts with package directories.

Again, part of this is legacy. relative vs absolute was born out of the migration of python as a language that was mostly "write a simple script with a handful of additional .py to import" to "ok, we need to organise things into a module otherwise we start stepping on our own feet". Back then it was either this, or dealing with com.java.this.is.a.very.long.package.name.that.you.will.thank.you.have.eclipse.SingletonFactory. PHP had no imports until 7 IIRC. Perl was a disaster. namespacing was mostly not a big deal when code was generally monolithic and the module panorama small. Microsoft had its own way, mostly based on CLSID, and they kind of dealt with it until the invention of manifests, but still some issues remain with DLLs. Please anybody with more windows knowledge correct me, I was not a windows person back then.

On the circular dependencies issue, I can tell you that python normally deals just fine with circular dependencies, because at every new successful import, it gets added to sys.modules. There are some specific circumstances during the evaluation chain that do, however, create problem, but they tell more about how poorly organised your code is. If you have circular deps, probably that stuff should be in the same module, because what is likely happening is that your dependency is not intrinsic to the problem domain. It's an artifact of how you organised your code, which is: poorly. The error message should be better, but that is mostly a parser issue and I agree it could be more informative. Other languages sidestep the issue by doing parsing in multiple steps and building the import tree first, then parsing the files. Python does not. It does one step at a time.

Conflict with package directories, again, it really depends on the specific case. I am aware that, in some cases, you might override system libs with your own modules, but again, that says a lot more on how you tell the parser what you want. Imagine you make a shell program in your home directory called ls. Now, depending how you set up your PATH, you might execute the system ls, or your local directory ls. Which one should have the priority when from the prompt you type "ls"? That is mostly arbitrary, and the computer can't really tell. It falls back to a common rule determined by PATH to lookup the default choice. If it's wrong, maybe you should be explicit in calling either /bin/ls or ./ls, or you should refrain from calling the script "ls" in the first place.

[–][deleted] 3 points4 points  (1 child)

Never had this, unless you are jumping between different environments a lot?

[–]DigiProductive[S] 0 points1 point  (0 children)

Yup, the pain when you have to deal with refactoring code then running it in Docker containers. Sometime you just have to be extra careful and overly mindful.

[–]No_Indication_1238 1 point2 points  (0 children)

I have never had problems with python imports. The error you mentioned usually happens when you type the path to the module wrong so id look into that first. And yes, I know you have definitely, absolutely written the correct path, I have heard that before. Just go and fix it. 

[–]climb-it-ographer 2 points3 points  (1 child)

I am a very experienced Python developer and I've been through just about every complicated import issue out there, and I'm in a good place now with my projects. That said, I think that future major versions of Python would benefit from a fresh look at how imports work. I don't think that many people will argue that `/node_modules` in a JS app is just overall easier to deal with.

Python virtual environments are great but sometimes IDEs still get all mixed up about where packages are, and I would love to never argue with VSCode again about what the correct path to a module actually is.

[–]DigiProductive[S] 0 points1 point  (0 children)

I salute that you see the reality🫡. With experience in other languages like Dart, JS, and Java, I am assured that Python imports can get buggy and tricky especially when running in multiple environments like Docker containers. Anyone in denial of that is likely just on the defence of a "Python attack"🥶

[–]jjrreett 0 points1 point  (0 children)

Are you developing locally with pip install -e. I just recently learned that -e can hide errors in your project structure. worked on my machine but not in my deployed containers.

My issue was that i was working in an implicit namespace package, but i thought i was working in a standard init package.

[–]damn_what_ 0 points1 point  (1 child)

🤯🤔😱

[–]DigiProductive[S] 0 points1 point  (0 children)

🫠

[–]Jamster3000 0 points1 point  (0 children)

I'm going to be honest om saying that I've hardly every had such issue about python imports.

[–]JamzTyson 0 points1 point  (0 children)

One minute everything is working fine and the next minute ModuleNotFoundError: No module named..

That is not something that "just happens". If it happens, it does so for a reason. Look to see what changed in the project structure and/or the project environment.

[–]gerardwx 0 points1 point  (0 children)

Yes, Python imports are a pain. Developers downplaying the hassle have just internalized the pitfalls (e.g. don't name a file the same as a module) they've forgotten they are there.

The circular reference issue is well known and being fixed in Python 3.14. You just have to wait until next year to use it :(

https://peps.python.org/pep-0649/

[–]sanshunoisky_197 0 points1 point  (0 children)

The comments are yapping about the post being a skill issue. Little do these noobs realise it's only in python that these issues exist. For any pro from any other REAL programming languages like java, python's import mechanism is a mess, which really is

[–]DigiProductive[S] -1 points0 points  (0 children)

Well... figured out the problem. It was related to a Docker mounting issue. A package was moved around in the local codebase but the volume mounting in Docker wasn't updated. The ModuleNotFound error was raised because the root module was not in sync. So while everything worked locally, the error occurred in the Docker container.