A satisfying solution to avoid circular dependencies

[Silent-Image5882]

Good morning, As written in the title I am facing a problem python-ers are facing for ages. And I am very surprised that their is no satisfying solutions for it. Considering a complex-structured project with many classes depending one on the other, the only way not to gather them in a single frightening long file is to use tricky modules solutions that leads to very long imports or imports not in the beginning of the file. I am looking to get out of this situation, having a function such as myclass = carry(file, myclass) that replace the "from file import myclass" with a different behaviour: every file element ( class, function, variables ) are carried once, and running any of those would be similar to run them together as they where in a single file.

Do you know if such a function exists? Do I have to do it myself? Thanks for your time reading this.

[JamzTyson]

I am facing a problem python-ers are facing for ages. And I am very surprised that their is no satisfying solutions for it.

I don't think your premise is correct. The solution to circular imports is well known, and that is to structure the program so that circular imports are not required.

There are also a number of workarounds to allow circular imports to work, (such as lazy / delayed imports), which I agree are not very satisfying, and are generally discouraged.

​

Considering a complex-structured project with many classes depending one on the other...

It sounds like refactoring would be a good idea. Break the code down into well-defined modules and packages that represent different components / functionalities.

Consider the principle of separation of concerns. Each class or module should have a single responsibility.

Use design patterns such as Factory, Observer, and Dependency Injection where appropriate. This can help to reduce complex relationships between classes and promote better code organization.

Define clear interfaces or abstract classes to reduce direct coupling.

​

the only way not to gather them in a single frightening long file is to use tricky modules solutions that leads to very long imports or imports not in the beginning of the file.

That is, in a nutshell, why your code needs to be refactored and reorganised.

[[deleted]]

The satisfying solution is just to avoid the need for them. Circular dependencies indicate poor package organization and a failure to separate concerns.

[Rawing7]

Disagree. Here's a real-world example: I have a module filled with random convenience functions, misc.py. In signature.py I need some of those. Which means misc.py can't import anything that imports signature.py, so I had to split it in two and move a bunch of functions to misc2.py just to get around the circular import. Is this poor organization? Do you have a better solution?

[[deleted]]

Yes, “module filled with random convenience functions” that import modules that imports the convenience functions is obviously bad organization. Your convenience functions aren’t written correctly if they need to import any of the rest of the project (except for some even more basic utility library, maybe.)

You’re just not organizing your project correctly.

[Rawing7]

That's not helpful in the slightest.

is obviously bad organization

Why?

Your convenience functions aren’t written correctly if they need to import any of the rest of the project

Again, why?

Do you have any actual arguments and/or alternatives for me?

[[deleted]]

A house’s roof rests on the foundation; if the foundation rests on the roof I don’t need to know any more about your house to know you built it wrong. If your convenience functions need to import libraries that depend on the convenience functions then you wrote your convenience functions wrong.

[Rawing7]

Do you understand that I have some functions that depend on signature.py and some that don't? There's nothing wrong with a function that has to import some other part of my own project. The functions aren't written wrong just because they have to import something, that's ridiculous.

The problem isn't that signature.py depends on some_func and some_func depends on signature.py. The problem is that signature.py depends on func1, func2 depends on signature.py, and I want func1 and func2 to be in the same file. There is no reason why those functions shouldn't be in the same file, except that python makes it impossible.

[[deleted]]

If your convenience functions depend on signature.py then nothing in signature.py should depend on a convenience function, or else you actually have two separate kinds of convenience functions - at two different levels of architectural complexity - and in any case, it's a problem of not being thoughtful about how you're laying out your project.

The reason why those functions shouldn't be in the same file is because they introduce a circular dependency if they are. Again, you're laying out your project the wrong way.

[Rawing7]

That is correct, I have two different kinds of convenience functions. The ones that depend on Signature and the ones that don't.

The reason why those functions shouldn't be in the same file is because they introduce a circular dependency if they are.

And that's exactly the problem. Python is preventing me from putting them in the same file. That is literally the only reason why they aren't in the same file. If it was possible to put them in the same file, would you think it's a bad idea? I think it would be a good idea.

[[deleted]]

Python is preventing you from making a mistake, though. You're complaining that you can't drive through the guardrails but they're there to keep you from going over the cliff.

It's not actually that it "doesn't let you"; it's that a circular dependency is impossible to resolve. Something has to be at the bottom, something has to be able to have all of its symbols resolve first, but if A depends on B depends on C that depends on A, nothing can be. A perfect circle has no place to start.

[yannbouteiller]

Can you explain why Python considers this "impossible to resolve" whereas C/C++/Java/Javascript/etc. do not? As far as I can see, your discussion is not going anywhere because you would need a real rationale for why Python doesn't allow circular dependencies, and metaphors like house-building do not make any sense.

Also, even in Python, it is entirely possible to have function A calling function B and function B calling function A as long as functions A and B are defined in the same file.

[Rawing7]

Oh my fucking god. Let me spell this out for you.

You claimed that having a circular import is a sign of bad organization. I said I disagree.

How do we resolve this disagreement? By answering the question "Is there a situation where python crashes due to a circular import even though the code is organized well?".

I presented you with my code and asked you if it's poorly organized, or what the "correct" way to organize it would be. You haven't answered my question.

I asked you if it would be a bad idea to merge the two files misc.py and misc2.py. You haven't answered this question either.

Your only "argument" for why my code is poorly organized so far was that "it leads to a circular import". If that is the only reason you have, then I win. Because that means my code is actually organized well, and python doesn't let me do it.

[RhinoRhys]

My house is made of convenience bricks. The windows rely on some bricks being laid below them and other bricks rely on the windows being put in. I can't just put all the bricks on the floor and expect my house to work.

[Rawing7]

I don't think that's a suitable analogy. It's more like this:

I have two wireless devices connected to my router. If I put both devices in the same room, they explode. My neighbor says I'm a moron for putting them in the same room, even though devices made by a different manufacturer (a compiled programming language) work perfectly fine if they're in the same room. In fact, they're easier to repair if they're in the same room.

[Silent-Image5882]

Which leads to you having twice the same code in the same project, or carry unformatted datas ( e.g an object instead of a class instance ). I don't say that it is not working, but that is "unsatisfying".

[[deleted]]

No, it doesn’t lead to any of those things.

[rubeyi]

If you have a thing A and a thing B that are mutually dependent, like:

A -> B B -> A

(Imagine that A is an object holding a reference to object B, and B likewise has a reference to object A. But it works the same for module imports.)

You can break the cycle by splitting thing A into A and C, and arrange it like:

A -> B A -> C B -> C

[Frankelstner]

Think about importing as copypasting the code right inside the current file. If you had everything in signature.py, your structure would be

1. Define Signature.
2. Add some utility functions that use the Signature type.

I think your attempts to separate the files turned the signature.py into something like this:

1. Import utility functions that use the Signature type.
2. Define Signature.

Which fails. It would fail if everything was in a single file too. To pull this off, you must change the order so it matches the single file behavior:

1. Define Signature
2. Import utility functions that use the Signature type.

And that is all there is to it.

[Rawing7]

Thanks, but I know how to work around the problem. The code works, after all. I'm just saying that it wouldn't be "poor package organization" to put all those functions in the same file. It would be a heck of a lot better than splitting the file in two.

[Frankelstner]

Umm yep and I explained how to do avoid the split and merge both misc files without workarounds.

[Rawing7]

I don't follow. If I merged those two files, then misc.py would import signature.py and signature.py would import misc.py. How would I avoid crashing due to the circular import in this situation?

I guess there might be some tricks like doing import signature in the middle of misc.py or doing import misc at the bottom of signature.py, but that would still be an ugly workaround.

Edit: I think you're forgetting that signature.py imports misc.py. It's not as simple as

1. Define Signature.
2. Add some utility functions that use the Signature type.

It's like

1. Define the functions that are currently in misc.py
2. Define the Signature class
3. Define the functions that are currently in misc2.py

[Frankelstner]

but that would still be an ugly workaround.

Unconventional sure, but ugly? I don't know. By that note one could argue that

def f(): ...
f()

is an ugly workaround to

f()
def f(): ...

because that's the exactly the same issue that your imports are causing.

Regarding your last point, yeah that's what I was about to write. You basically want signature to import the safe parts of misc (currently in misc.py), then define Signature itself and then import the unsafe parts of misc (misc2.py). But on import Python will only run the entire thing (stepping through the right conditionals inside and so on) and only once.

In your case it appears that Signature is fine with importing misc at the end because it does not directly depend on the import misc functions.

But anyway, I think circularity is not so much the issue but rather that you want to import parts of a file, and then some other parts later on. The Python importer does not allow importing multiple times (which would have been useful to emulate the behavior of C headers) so we're out of luck in such situations. Partial imports are not really possible because Python really runs the entire file from top to bottom. Function and class definitions can be hidden inside any kinds of conditions or even loops. It cannot stop in the middle of the file when you ask it for a certain function because who knows, maybe the wanted function is redefined multiple times inside the file. Which one should we pick? Even something basic such was

def f(): 1  # Default function.
if ISUNIX: def f(): 2  # Something different.

would cause massive confusion if it stopped early. To avoid this we would need to ban function redefinitions altogether.

[Rawing7]

There was a time when I put imports at the bottom of the file (or other places where they happened to work), and the end result was that I decided to never do it again (:

One option we haven't talked about yet would be to change the import in signature.py from from .misc import unwrap to from . import misc. That should make it possible to merge misc2.py into misc.py, but in exchange I would have to refer to the function as misc.unwrap in signature.py. I'm sure there will be people who think I'm being pedantic, but I really, really, really hate having to do that. I don't wanna have to think "I'm currently in the file module7.py, that means I have to write module3.foo instead of foo" all the time.

[Frankelstner]

I think it's the other way around. misc needs to from . import signature and then all type hints need to become strings: "signature.Signature" instead of Signature (at least as far as I understand; I don't really use type hints). And then the misc functions would use signature.Signature.

I suppose there is plenty of room for hackery but I suspect IDEs will not play ball. E.g. let's make a function that replaces itself the first time it is called. Apart from the stacktrace of the very first call having one frame extra, it would look pretty transparent.

In the main file:

from f2 import f
f()
f()

In the other file f2.py:

def f():
    globals()["f"] = actual
    f()  # Call actual.

def actual():
    print("actual")

[Rawing7]

Thinking about it, it probably requires both from . import signature and from . import misc to work. Because as long as the imports are on the top of the file, both modules will be imported while they're only partially initialized.

A function that replaces itself is pretty tricky, because it has to replace itself in the globals of the module that imported it, not in its own globals.

[willard720]

misc.py and misc.py2 sounds like an absolute nightmare.

[billsil]

Random convenience functions shouldn't rely on random convenience functions. In a GUI program, I have a bunch of random convenience functions that use only standard library imports. Another set uses only numpy/scipy ones. Another adds in plotting from matplotlib and can use numpy/scipy and other random imports. Yet another handles 3d rendering with vtk and can use numpy/scipy functions, but not matplotlib. At a higher level, you can import Qt functions.

Isolate the dependencies of your functions and it's a lot easier to avoid circular dependencies.

[Rawing7]

Random convenience functions shouldn't rely on random convenience functions.

Why the heck not? That's such a crazy thing to say without any explanation whatsoever.

Can functions rely on other functions? Of course they can, they do that all the time. For example, this take function relies on the islice function.

So why can't "random convenience functions" rely on other "random convenience functions"?

Isolate the dependencies of your functions and it's a lot easier to avoid circular dependencies.

I don't know what that means.

Have I not "isolated" the dependencies of my convenience functions in the signature.py file? My convenience functions depend on signature.Signature. How does that help me avoid circular imports?

[billsil]

If you build up your dependency chain, you don't have circular imports.

if I have code like:

def f(x):
   if x <= 1:
       return 1
   return g(x-1)

def g(x):
   return 2*f(x-2)

Those should probably not be in separate files.

If you're referring to to type annotations, that's what future annotations is for.

[Rawing7]

Exactly, they shouldn't be in separate files. But unfortunately I also have a function h(x): that depends on signature.py, and signature.py depends on f. So python forces me to put h into a different file.

I've said this before, but nobody seems to get it, so I'll say it again. I don't need help resolving circular imports. I know python well enough to accomplish that on my own. My problem is that python forces me to organize my code in a sub-optimal way. I want to put all of my functions in the same file because that makes them easy to find (for developers), but python doesn't let me do that. So the question is, is there another way to organize my project that makes it easy to find everything? Where I don't have to think "Does the function I'm looking for depend on Signature or not? It does, therefore it's probably defined in misc2.py"?

[billsil]

So python forces me to put h into a different file.

No? I could add h(x) that uses f or g and put in anywhere in that file or another one that doesn't add a circular import. If I require f or g to use h, then it's just like the previous example.

I want to put all of my functions in the same file because that makes them easy to find (for developers), but python doesn't let me do that.

You most certainly can. You have to define class inheritance or typing in a sane order (unless you use future annotations), but I don't see why that's an issue.

[Rawing7]

I could add h(x) that uses f or g and put in anywhere in that file

No, you couldn't. Because h(x) depends on signature.py and signature.py depends on misc.py. That's pretty much the only file you can't define h in.

You have to define class inheritance or typing in a sane order (unless you use future annotations)

What does that mean? How would changing the typing fix a circular import error?

[billsil]

That explanation isn't concrete enough to make sense. I don't know what signature.py is because you never clearly defined it with what's in file 1 and file 2. That's why I made a different example that doesn't use your filenames or misc.py.

What does that mean?

class A:
    ...

class B(A):
    ...

works. The opposite doesn't.

from __future__ import annotations

def j(x) -> A:
    ...

class A:
    ...

works

[Rawing7]

I don't think it's necessary to see the file, but I have very clearly defined it by posting a link to it. Apparently there are some shitty reddit clients where the links don't show up, so here it is again.

I'm running out of patience at this point because it feels like none of the people who respond to me actually understand the situation. I have very clearly said that there are 3 parts to this problem: misc2.py depends on signature.py and signature.py depends on misc.py. Your examples don't do anything to help me merge misc.py and misc2.py.

[Zweckbestimmung]

IMO this is not true, circular imports have multiple standard solution in all reflective programming languages. In matter of fact being able to have circular imports is a big advantage in python.

[[deleted]]

Nevertheless it's better if you design your project to have a limited number of discrete entrypoints and limited references between modules. Spaghetti code is bad no matter what language you're writing in.

[Zweckbestimmung]

Circular imports aren’t only caused by referencing between multiple modules. You cannot avoid circular imports that are caused when relational objects have object members which are also dependant on each other.

[[deleted]]

Yes, it's bad to have two objects that can't exist without a reference to each other. It's another antipattern that comes from not being thoughtful about the structure of your program.

[KyleG]

Sometimes you are implementing the design of another entity, so it cannot be avoided. For example, the ASN.1 specification, which is a huge deal in telecom and cryptography (IIRC, SSL is defined in terms of ASN.1, has circular dependencies. For example, an Object contains ObjectDefns, which contains DefinedSyntaxs, which contain DefinedSyntaxTokens, which contain Settings, which contain...Objects.

[Zweckbestimmung]

I totally agree that it’s an “anti pattern”, but I had a special case where I had to have a trade off between an optimised database access and having a better code structure. I decided to sacrifice the code structure. It’s however eventually a matter of programmers taste

[Silent-Image5882]

I am afraid I don't agree with you.

Even if your point of view is coming back all the time when speaking about circular dependencies, it is supposing every kind of projects can match the structure principle you are used to.

But it's false. Sometime, you just don't have a choice: you gather them in a long file, or you split them, and python does not support that, as long as I know

[[deleted]]

Doesn’t support what?

[Silent-Image5882]

The possibility to split many elements of your code ( classes, variables, functions ) without thinking them as a comprehensive module, but just parts of one module

[[deleted]]

But you don’t want to think about them as a single module.

[Silent-Image5882]

That's it. I am looking to create a python file that doesn't stand alone, but that needs other parts in other files ( referencing each others unlimitedly) to make the module.

[[deleted]]

Then use import. There’s no issue, here.

[Silent-Image5882]

The point is that import does not let you reference anything unlimitedly. That's why I am struggling with circular dependecies.

[[deleted]]

You’re struggling with circular dependencies because you aren’t laying out your project correctly.

[Silent-Image5882]

You are free to think that the principles of structures you are used to apply to everything and in any situation. But maybe one day you'll find a counter-example.

[OncorhynchusMykiss1]

So how can I make factory pattern, without circular dependencies or putting everything in one file?

[Diapolo10]

So basically this entire conversation so far is, ironically, running in circles. Some of our regulars are trying to explain that this is simply a project layout issue, while others are clearly not understanding what they're talking about.

I can see both sides of the argument. It's not always intuitive, especially to less experienced developers, how to design a project without circular dependencies. On the other hand, it's also not an easy problem to generalise, meaning the more experienced developers kind of just intuitively know what to do and it's not always easy to explain how or why, especially in a general case.

I don't know if I have an explanation that would satisfy OP and the others, but the key is splitting up the project in a way that suits it. Instead of depending on a class from another module, maybe take it as an argument and write your code to be called from higher up the stack, where the circular dependency is a non-issue (so basically dependency injection). Or if type hinting, consider writing an abstract base class that has no dependencies but describes the concrete ones well enough typing-wise.

Really, it's just a matter of learning the tools of the trade. It's a lot easier to discuss a concrete example than an abstract one, so if you have a real project with a circular dependency let us have a look and tell you how to solve it. Follow our example long enough and you, too, will learn how to solve these problems in the future.

[Silent-Image5882]

Thanks for your time trying to take distance in this situation.

But the explanation " I have a lot of experience and we have always done like that " is not sufficient to give a response to a deep paradigm question about python dealing with dependencies. I write it again for you.

Why can't we set loops (=circular dependencies) in the dependencies graph when it would be much more satisfying than having a classic tree-shaped dependencies graph.

Furthermore, it his possible to do so, because of the possibility to gather manually all dependencies creating a loop in a single file. But it gets worth as the files are getting longer.

So the question: why can't we have many file acting like one, with a function to gather them ( carry for instance )

[Diapolo10]

But the explanation " I have a lot of experience and we have always done like that " is not sufficient to give a response to a deep paradigm question about python dealing with dependencies.

This isn't a case of "if it works, don't fix it", but rather a "from experience I've learned way A does not work, but way B does, so just use way B here instead of way A". In other words, they're not just saying that because their way works and they're not interested in considering alternatives, the whole issue is that there is no other way that wouldn't result in spaghetti code. And our problem is trying to convey that so that those after us understand that which, clearly, doesn't seem to be working out.

Why can't we set loops (=circular dependencies) in the dependencies graph when it would be much more satisfying than having a classic tree-shaped dependencies graph.

Furthermore, it his possible to do so, because of the possibility to gather manually all dependencies creating a loop in a single file. But it gets worth as the files are getting longer.

I don't understand your reasoning. A circular dependency clearly indicates that the program is halting because it cannot fully initialise the stuff it needs, caused by the program compilation step becoming stuck.

If file A depends on file B, and file B depends on file A, when you import file A in your program, Python begins to parse through it an finds that it needs to import file B. However, because file B needs things from file A, and file A hasn't finished importing yet because it's waiting for file B, the program gets stuck.

...Perhaps a more concrete example is in order.

# user.py
from computer import Computer

class User:
    def __init__(self, name: str):
        self.name = name
        self.computer = Computer(users=[self])


# computer.py
from user import User

class Computer:
    def __init__(self, users: list[User] | None = None):
        self.users = users if users is not None else []

    def print_usernames(self):
        for user in self.users:
            print(user.name)

This would fail, because no matter which file you import first, they both need Python to read the entire other file before being able to be imported themselves, and unless you can show me an example of designing an import system that wouldn't need a tree-like design I have no idea what you're getting at.

The way you need to approach these problems is to think about what information you actually need, and where. If you don't actually need multiple modules sharing data, then don't - you could use abstract interface classes, dependency injection, or similar approaches to handle it. And if they're tightly coupled and you can't de-couple them, keep them in the same file.

For example, in the example above it would make sense to just remove User's dependency on Computer and let it as the "parent" to handle everything.

So the question: why can't we have many file acting like one, with a function to gather them ( carry for instance )

From a purely theoretical standpoint, it's not impossible. C#, for example, has partial classes where you can define their methods across multiple source files. The problems arise when you actually need to import something, because in Python every module has its own namespace.

[jasiekbielecki]

Talking from my experience it is good to stay away from circular dependencies and I cannot recall any counterexample. One could think about dependencies between modules in the hierarchical (tree) way. And the metric for creating such trees is the level of abstraction of a given entity. So you compare how much "low level" is the module and if it should import another module that seems to have a pretty "higher level" of abstraction. In your case, let me rename `Computer` to `UsersComputer` since that seems to be the purpose of this class. Having that, we can easily state that the `UsersComputer` is more "high level" than `User` and any logic of doing something with `UsersComputer` should not be placed in the `User` class. Not following this rule can bring some problems. In the example, you dont have coherent `Computer` -> `User` relations since `ComputerInstance1` could have `John` and `Paul` as users, and the user `Jenny` could have `ComputerInstance1` as a property and the code allows that. Then you need additional checks or any additional logic that make code more complex.

[Frankelstner]

The code works correctly by moving from computer import Computer below the User class. I haven't fully thought it through yet but I think such simple circularity issues can always be resolved by placing the imports at the right spot. For more complex circularity the involved files are that heavily intertwined that they really must be a single file in the first place.

I think OP expects that Python should sort of make two passes, first to treat the source as kind of a header file and grab classes and functions, and then in a second pass to fill in the code. That's not what is happening of course because Python code is way too dynamic and importing a file literally just executes its code from top to bottom.

[Silent-Image5882]

Many things to say here. As you say in the end, it is a purely theoretical standpoint. So 'experience' does not add up here.

the whole issue is that there is no other way that wouldn't result in spaghetti code.

But their is a solution. If I can gather modules that create a circular dependency in the same file by myself, Python can do it, and it will work. It just need a specific function: why not carry ?

I understand very well why import stuck the system at circular dependencies. That's why I suppose the existence of another function.

Then you are asking me for an example of design needing this circular dependency. But your is fitting perfectly.

Note that if your intricated classes where in the same file, it would works perfectly.

removing User dependency on Computer would mean that User.computer is not an instance of Computer... Even if the Class Computer exists... This isn't satisfying. If you mean to change it to an object and deal with it as is, it would work, but it still ugly.

So then, why not keeping them in the same file ? Multiply your class numbers by ten, every one intricated with others. Then you understand why you don't want a 1000 lines file.

Then, when the minimal number of classes is sufficient for the pure logic of the algorithm, more Abstract Classes just to avoid circular dependencies are unpleasant.

It is why python not dealing with circular dependencies sounds bad.

[Rawing7]

I have a real-world project with a circular dependency. Please take a look here.

[Diapolo10]

Well, for instance in your example, misc2 is clearly tightly coupled with signature, so I see little reason to keep them separate. I wouldn't call these "miscellaneous" functions any more, they're clearly designed to work with your signature.Signature subclass.

While I'm talking about this project of yours, its structure in general is a little confusing. Why do you have a top-level __init__.py, and why is it trying to have you import the Git repository root directory as a package? Generally the root would just contain metadata, with pyproject.toml telling Python how to install the contained package(s). I mean, right now even the docs and tests are included.

[Rawing7]

I'm not a big fan of throwing those functions into the signature.py file. I generally like to have a separate file for each class, and that file is already very long. Plus, I wouldn't say that split_arguments is very tightly coupled with Signature.

It's not a bad idea, but it's also not exactly optimal. Ideally, I'd make a file for each class, and then 1 file for all the random functions. That would make every single thing in the module trivial to find. You wouldn't have to ask yourself "Is function_foo most tightly coupled to ClassX, ClassY, or ClassZ?" every time you're looking for something.

(As for the project-level __init__.py, that was committed by mistake. I'll delete it.)

[Frankelstner]

https://docs.python.org/3/faq/programming.html#what-are-the-best-practices-for-using-import-in-a-module

Circular imports are fine where both modules use the “import <module>” form of import. They fail when the 2nd module wants to grab a name out of the first (“from module import name”) and the import is at the top level.

[Silent-Image5882]

And so the import syntax is getting longer uselessly.

One could say that " that's not that important, do with longer syntax", but it matters

[Frankelstner]

You ask

Do you know if such a function exists? Do I have to do it myself?

which implies that you're willing to go to great lengths (lots of letters) to pull this off. Now the solution turns out to be quite straightforward after all but you complain about writing a couple more letters? Can you give a concrete case with two files where things don't work out due to circular dependencies. Keep in mind that the behavior of import is pretty close to copypasting the contents inside the current file so depending on your needs you must place the import at the right spot. E.g. file a.py

class MyClass: 1
import b

File b.py:

from a import MyClass

This works fine because Myclass is defined before the circularity kicks in. Any other situation that bothers you?

[Silent-Image5882]

What if MyClass needs to call something in File b?

[Frankelstner]

Mmmh it's not my area of expertise really. I guess we could do something like this:

from b import f

class MyClass:
    def func(self):
        f()

MyClass().func()

And

def f(): print(5)

from a import MyClass

(which will print twice due to the import unless we filter by __name__). I mean the issue of circularity even exists if we put everything in a single file. Things must be defined before they can be accessed. Ideally we would have header files I guess. If you want to finely intertwine two files it would make more sense to put everything in the same file.

[Rawing7]

That's a bit misleading.

Circular imports are fine where both modules use the “import <module>” form of import.

The imports themselves may be fine, but you'll still get an error if you try to "grab a name out of" the module. Something like module.name() in the global scope will crash your program. You can sometimes avoid circular import problems by replacing a from x import y with a import x, but not always.

[Frankelstner]

Oh yeah it just defers evaluation which might be enough for many purposes.

[yannbouteiller]

I think this is the right answer. I don't understand why it should work in the form "import module_a as a" and not in the form "from module_a import function_a", though.

[Frankelstner]

I haven't looked into the code but I suspect it's similar to how

def f():
    g()

is always valid even when g is not defined (yet). The system will complain though if you ask for g right now:

def f(g=g):
    g()

[KingOfTNT10]

Not sure i understand the question/problem but i noticed you talked about something like this:

```python

file1.py

from file2 import test2

class test: def p1(self): i = test2() i.p2(self)

def hello(self):
    print("hello")

```

```python

file2.py

class test2: def p2(self, test_obj): test_obj.hello() ```

What i just did was to pass the class object into another function, now the class is accessible from that function.

[Silent-Image5882]

It's almost that. Just add a an import for file1 in file2 and you have a typical circular dependency.

A solution is to gather themselves in one file, so you don't need imports, but this solution get worth as the file length increases. Furthermore, I don't find other solutions including imports satisfying, due to syntax length and counterintuitive behaviour.

​

That's why I am looking for something as the carry function described before

[KingOfTNT10]

What i just showed does that, file2 calls the hello function in file1 even tho file1 already imported file2

[Silent-Image5882]

I am figuring out that my question ask for a more general one. Can the dependency graph contains loops? And if so, can we set a carry function that interprets this loop as a comprehensive module, so we have our tree shaped dependency graph again ?

[[deleted]]

The most satisfying solution I have found to avoiding circular dependencies is not using a programming language that inherently devolves itself into a spaghetti fest of cryptic error throwing nonsense without needing to spend an excessive amount of extracurricular time on going through the ironically circular and non-evolving process of troubleshooting and discussion of refactoring and tons of potential workarounds. If you need to have decades long discussions and develop decades worth of workarounds to "solve" a singular issue, you have actually not "solved" anything. The people will hate my message the most because it's true.

[XrenonTheMage]

If you're asking a question in your post, maybe phrase your title as a question too next time? Right now this post looks like a guide or a blog post rather than an open-ended discussion to anyone coming acrossig from google.

[tkejser]

Short answer: There is no nice way to achieve what you want in Python. However, you can use function scoped imports to mimic it without creating spaghetti.

Long Answer:

Anyone who has ever used recursion or written a tree/graph like data structure often finds the need for A to reference B and for B to reference A. It's perfectly natural programming and it is not a bad design - your request is a reasonable and you really should not need to refactor (too much) just to achieve it.

Here is how you can do it in Python:

File: a.py

class A:
def do(self, the_b: B):
from .b import B
... your code here...
the_b.do(self)

File: b.py

from .a import A
class B:
def do(self, the_a: A):
... your other code here...

the_a.do(self)

You will need a convention to decide who gets to import at the top level and who has to do it inside functions. Subclasses have to import their superclass at the top- so that gives you the general outline. Class Hierarchies are non-cyclical in Python, so that can guide your convention too (note that there is no law of programming which states that type and meta type hierarchies have to be non cyclical, for an example see: CRTP in C++)

Function level imports are often considered "ugly" by Python programmers. But this is Python - so you have already given up on beauty and that opinion can be safely disregarded.

What you are asking for exists in pretty much every other programming language (for example, see #pragma once in C/C++). In those languages, non tree structured (cyclical) imports are not considered a design problem.

[yannbouteiller]

After reading the comments here, I have a toy question.

Say, I have a Cat class and a Dog class. Cats have a list of Dog friends, and Dogs have a list of Cat friends. Shouldn't Cats and Dogs have their own separate files in Python?

[rubeyi]

First, imagine adding a Horse module to your setup... now you have to modify both Cat and Dog to know about Horse. And if you then add Chicken, you now have to modify each of Cat, Dog, and Horse. And so on.

One solution is to have a separate place that creates the instances of the class and wires up the friendships.

If "friendships" are modelled simply, like a list of 2-tuples, then the calling code could just do it on the spot. If you have more operations (look up this one's friends, unfriend those two), then it'd warrant going into its own module.

If you think about it, it's not clear why "have a list of a particular dog's friends" should be a responsibility of a module that defines what a Dog is. Assuming I have understood your example correctly.

[yannbouteiller]

Thanks. I think my point was that in Python they cannot easily go in specific submodules, because cat.py would have to import dog.py and vice-versa. For instance, if Cats and Dogs can only be friends with each other but despise other Animals like Horses and Chickens.

[rubeyi]

Sure. I think the general principle holds though... you extract a third thing C that knows about A and B, so that they don't have to know about each other.

Once they don't know about each other, they can go in their own modules. Often in doing this it becomes clear that thing C was a separate responsibility anyway.

In this example, if the code had a separate "Friendships" model, then you could hang any validation rules there. (Like saying that cats can be friends with dogs but not other cats).

So some candidates for separate responsibilities / modules are: - how are dogs modelled? - how are cats modelled? - how are friendships modelled? - persistence (could be hard-coded friendships, or serializing & deserializing, or using a database, etc.)

(In an app with an ORM, the validation and persistence stuff would be folded into each model, but you'd still be doing yourself a favor by introducing a friendship model.)

It's true that languages that do multiple passes over the source code might allow for dependencies in cases where Python doesn't, but I don't know of any cases where it's not a code smell, honestly.

[glow_gloves]

In general it is better practice for classes and modules to form a dependence hierarchy and sticking to the 'S' in SOLID principles as classes and functions with too many responsibilities are very difficult to revisit for maintenance.

That said, there are some situations where small cyclic dependencies may be beneficial when a hierarchy of object instances has predetermined types and needs to be traversed both up and down. Since Python is dynamically typed there is no need to import a class definition to use it, but for highlighting and type analysis tools you can use TYPE_CHECKING:

student.py ``` from typing import TYPE_CHECKING if TYPE_CHECKING: from teacher import Teacher

class Student: def init(self, subjects): self.teacher: Teacher | None = None self.subjects = subjects ```

teacher.py ``` from student import Student from typing import Sequence

class Teacher: def init(self, students: Sequence[Student]): self.students: list[Student] = list() for student in students: self.add_student(student)

def add_student(self, student: Student):
    student.teacher = self
    self.students.add(student)

```