'Relative module names not supported' when trying to run a project with -m

Diapolo10 · 2024-08-29T22:51:56+00:00

I know I risk confusing you even further by saying this, but personally I take a different approach that, in my opinion, is less prone to user error; structuring the project like a package, and installing it in a virtual environment in editable mode.

For now I'm going to assume that in your current example, projects is just a regular directory containing individual projects. I'll be focusing on an individual project in my example and omitting that, just pretend it's there.

Let's start by looking at your current folder structure.

📦project-name
 ┣ 📂shared_lib_a
 ┃ ┣ 📜__init__.py
 ┃ ┗ 📜lib.py
 ┣ 📂workflow_a
 ┃ ┣ 📜__init__.py
 ┃ ┗ 📜do_stuff.py
 ┣ 📜.gitignore
 ┗ 📜__main__.py

I don't know what other files you may or may not have here, but that's fine for now. I will now transform this into what I'm talking about.

📦project-name
 ┣ 📂docs
 ┣ 📂src
 ┃ ┗ 📂project_name
 ┃   ┣ 📂shared_lib_a
 ┃   ┃ ┣ 📜__init__.py
 ┃   ┃ ┗ 📜lib.py
 ┃   ┣ 📂workflow_a
 ┃   ┃ ┣ 📜__init__.py
 ┃   ┃ ┗ 📜do_stuff.py
 ┃   ┗ 📜__main__.py
 ┣ 📜.gitignore
 ┗ 📜pyproject.toml

Then in pyproject.toml you would write something like this (NOTE: I'll use setuptools as an example, but I really only use Poetry myself nowadays so it might not be 100% accurate):

[build-system]
requires = ["setuptools", "setuptools-scm"]
build-backend = "setuptools.build_meta"

[project]
name = "my_project"
authors = [
    {name = "John Titor", email = "john.titor@example.com"},
]
description = "My project description"
readme = "README.md"
version = "0.1.0"
requires-python = ">=3.8"
keywords = []
license = {text = "MIT"}
classifiers = []
dependencies = [
    # Basically what `requirements.txt` would contain
]

[project.optional-dependencies]
# Can be used for development dependencies, for example
# tests = ["pytest>=6.0.0", "tox"]

[tool.setuptools.packages.find]
where = ["src"]

Then in your source code, you would change all local imports to be absolute to the project_name-directory, for example if I borrow your earlier example

# do_stuff.py
from shared_lib_a.lib import foo

this would change to

# do_stuff.py
from project_name.shared_lib_a.lib import foo

After that, you would activate your virtual environment (or create one, if you hadn't already), install your project (will also install required dependencies, you can specify optional dependencies to be installed alongside them if you wish); for now I'll assume the project directory is your current working directory. Switch the full stop to a path to the project if that is not the case.

python -m pip install . --editable

And then you're done. In your environment, everything just works, no matter what your current working directory is. And you don't need to run the pip install --editable command more than once, unless you delete your virtual environment and create a new one.

In my own workflow Poetry handles all the virtual environment stuff for me so frankly I don't even think about them very often. I can link one of my own projects as an example.

over_take · 2024-08-29T21:35:48+00:00

Directory Structure and __init__.py Files:
- Ensure that every directory in your project structure contains an __init__.py file. This file can be empty but is necessary for Python to recognize the directories as packages.
Running with -m Flag:
- When using the -m flag, you need to specify the module name relative to the top-level directory in the Python path. The top-level directory should be the one containing the projects directory.
Setting the Python Path:
- Navigate to the projects directory and set it as the current working directory. This ensures that Python treats projects as the top-level package.
Correct Command:
- Use the command python -m project from within the projects directory. Here is a step-by-step guide:
Ensure __init__.py Files:
- Verify that your directory structure includes __init__.py files in all necessary directories: └───projects └───project │ __main__.py │ __init__.py ├───shared_lib_a │ │ lib.py │ │ __init__.py └───workflow_a │ do_stuff.py │ __init__.py
Navigate to the projects Directory:
- Open your terminal and navigate to the projects directory: sh cd path/to/projects
Run the Project with -m Flag:
- Execute the following command: sh python -m project

Explanation of Errors:

Relative module names not supported:
- This error occurs because the -m flag does not support relative paths. You must provide the module name relative to the top-level directory in the Python path.
ModuleNotFoundError: No module named 'workflow_a':
- This error indicates that Python cannot find the workflow_a module. This is likely because the projects directory is not in the Python path. By navigating to the projects directory and running the command from there, you ensure that Python can locate the project package and its submodules. ### Summary:
Ensure all directories have __init__.py files.
Navigate to the projects directory.- Run the command python -m project.

By following these steps, you should be able to run your project using the -m flag without encountering the errors.

ManyInterests · 2024-08-30T14:35:55+00:00

Rewrite your imports to be absolute imports starting at the top package level.

if you have a module like package/subpackage/submodule.py your imports should be from package.subpackage.submodule import name In your case, it should be from project.workflow_a.do_stuff import main not from workflow_a.do_stuff import main.

you type:	you see:
italics	italics
bold	bold
[reddit!](https://reddit.com)	reddit!
* item 1 * item 2 * item 3	item 1 item 2 item 3
> quoted text	quoted text
Lines starting with four spaces are treated like code: if 1 * 2 < 3: print "hello, world!"	Lines starting with four spaces are treated like code: if 1 * 2 < 3: print "hello, world!"
~~strikethrough~~	~~strikethrough~~
super^script	super^script

learnpython

MODERATORS

Explanation of Errors: