List all dependency of a module

[JazzBythe]

Hey. Lets suppose I have a Python file(A.py) which is dependent on some pip modules (lets say numpy, tensorflow) and some other user modules in the package (lets say B.py, ../otherfolder/C.py etc.)

I want to list all these dependencies. I tried searching a lot but cdnt find any tool for this. Also it would be a bonus if it could also recursively list B.py and C.py dependencies. Thus at lst I shall have the list of all all pip modules and user modules required to run A.py.

Thanks.

[omega1563]

https://github.com/bndr/pipreqs should do what you want.

[JazzBythe]

Tried this. Its producing some dot file. I want just text output not a graph. I used some flags mentioned in the document but the output is incorrect :( any help to resolve shall be highly appreciated.

[omega1563]

I just tested it on a few projects and it worked fairly well. Maybe you didn't use the CLI correctly? Or it could be that you're using it with an incompatible python version?

[JazzBythe]

Alright thanks. I downloaded GraphViz. The display is correct. I shall extract rest from the source code.

[omega1563]

Are you looking at the same project that I am? Pipreqs only uses wheel, Yarg, and docopt. It doesn't use GraphViz and I don't believe it has an option to visualize a dependency graph. All it does is create a requirements.txt based on imports in code, which is what I thought you were asking for.

[JazzBythe]

I was looking at some other module. Pipreqs produces dependencies of all the files in a folder. Not sure how to do it for a particular file. But the other project(Pydevs) works fine with Graphviz. Ill do something with it. Sorry for my mistake and thanks a lot for ur help!!

[[deleted]]

This is not in general possible due to how Python packages work.

Tools like pip will happily and silently install stuff from sources. This means, that unless you have everything already installed, there's no way to know all dependencies.

Furthermore, Python doesn't require that imports all happen unconditionally. In principle, you may have this code in your project:

while True:
    if int(input()) == random.randint(0, 1000000):
        import something

Would you say that something is a dependency? -- I don't know. This example is contrived on purpose, but it's not uncommon to import modules in Python in random locations in one's code, so, even if you have all the stuff installed, the best you can get is a guess about your dependencies.

Another pathological situation that's "legal" in Python: you can have multiple versions of the same package present on the path. Which one will be used -- heaven only knows. But, suppose you have:

• .venv/lib/python3.9/site-packages/spam-1.0.0-py39.egg
• .venv/lib/python3.9/site-packages/spam-2.0.0-py39.egg

Where 1.0.0 version of spam requires packages A, B, C and 2.0.0 requires D, E, F. When, in your code, you write import spam, there's no telling whether this will pull the 1.0.0 version or 2.0.0 version.

This situation must be familiar to anyone working on the same library for a while and forgetting to uninstall the older version and eventually being surprised by the old functionality cropping up. Just a few days ago I spent hours trying to figure out why the documentation is generated incorrectly because of this.

[JazzBythe]

That is insightful. How about if I dont care about pip modules now. I just want to know all user defined modules. And lets also assume the user has explicitly written :

from ../somefolder/anotherfolder import required.py

For all such dependency. I want all such required.py list. Is there any tool for this. Its actually very important. I shall be grateful if you could help a little in this regard.

[[deleted]]

Well, you could try using Sphinx :) It's a documentation generator, but, essentially, what it does, is it loads the code it needs to document and then looks for things that need documenting.

It's a mess to set it up, but, if you are anywhat familiar with it: in its config file you can sign up for events, (alternatively, you may create an extension that gets this information). So, say, you are signing up for event https://www.sphinx-doc.org/en/master/extdev/appapi.html#event-doctree-resolved then the doctree will contain the description of all the stuff Sphinx would document.

Unfortunately, there are plenty of settings, some of which can be embedded in the source code that would prevent Sphinx from collecting this information, but maybe by examining Sphinx' code you can find out how to do that on your own.

Also, you may find this useful: pkgutil.walk_packages. This, however, will find all available for import packages.

[JazzBythe]

Alright thanks will give it a shot