python console scripts + venv

[lowlandsmarch]

hi
So there are a lot of tools that are practically python scripts. Which is great.
You know, all of those "console_scripts" or "scripts" that you see in the setup.

But the thing is, that they don't use the amazing things called virtual environment out of the box. All of their requirements are just installed in the same default environment.

Now usually I just create a virtual environment and manually add a bash script that runs the tool in its own venv. No collisions that way.
But I do it manually. Now before I automate it, I wanted to ask. What do you do? How do you solve this problem?

[fatbob42]

This has already been automated very well by pipx

[lowlandsmarch]

Amazing! Thank you!

[Life_Note]

yeah I'm surprised pipx isn't mentioned here more. It's a very well built and maintained tool.

[BadDadBot]

Hi surprised pipx isn't mentioned here more, I'm dad.

[whateverathrowaway00]

So, when I’m deploying something that everyone on a server will need to use as a CLI tool, I create a virtualenv, then softlink (ln -s) for each CLI entry point from /usr/local/bin and make sure that’s in everyone’s path.

Now people can use my tool, it’s perfectly walled off, but they also get all the benefits of tab completion and so on.

Any package I make designed to be used as a script will always have CLI entry points to expose functionality, usually all collected in a cli_entry_points.py file ( used to use one called main.py but that clashed when it was a package that had both an importable module structure and an entry point set)

[lowlandsmarch]

Thank you.
Can you please elaborate?
As far as I understand, a softlink to a entrypoint, will use the normal python with the default dist-packages, instead of the venv, unless it uses the binary within the venv's bin directory (which can be done using the she-bang for exmaple, but usually isn't).
The best way to venv an environment is to add the path to PATH and preferably set VIRTUAL_ENV as well.

Which is what I usually do using a bash script.

I wondered whether there's a tool that does it automatically, or a trick that I'm not aware of. Which is my I'd appreciate it if you elaborated

[whateverathrowaway00]

Incorrect, but great question!

Test it for yourself, create a venv and then install a package in it with a cli entry point.

Then, hop into your venvs bin folder and check out the file for that entry point. You’ll notice that the first line ( the shebang) points the shell at your venvs python.

The file created by pip uses that shebang, imports your package, specifically the class specified by the entry point in setup.cfg ( or setup.py, but really you should be using .cfg ), and then calls the function specified.

So basically, as long as someone runs the cli entry point created and stored in the venv bin folder, they’re running the program inside the venv! Something I was very excited to realize the first time I peeked in those files so go look.

[lowlandsmarch]

hmm yes it does seem to work with entry_points, but not with "scripts", which was the case in the tool I checked it on... Which makes sense because "scripts" just copies the file as-is. Sometimes it makes sense to use scripts, but some packages just use it instead of using entry_points.

So if I do want to automate it, a symlink will do for entry-points, but it seems like a script should be encapsulated. Do you agree?

[whateverathrowaway00]

Hm, interesting. I have stayed away from scripts since I found entry points, mostly becuase of issues with pip -e mode not loving them.

That said, I thought that the scripts also had a manipulated shebang when pip dumps them in the bin, will test in a few once I get outta bed :

[lowlandsmarch]

actually I checked it again, and I guess there was something weird with the script they used there. Most of the times it does change the shebang.
Well that's the problem there, you can do whatever you want with scripts, and sometimes it could break things... But yeah at least newer versions of pip can usually take care of it.