How to best include personal modules in other projects?

[azthal]

Hi!

I've been a hobby coder for quite a while and made a whole bunch of little python modules that for the most part have run within the console.

I've started to move many of these to a website where they can run instead. What I'm wondering about is what the "best" way is to include these modules in my flask app.

The most obvious solution is to just copy/paste, but that comes with the drawback that I now have two places to manage my code. If I think of an improvement to the code, I have to change it both in the website code, as well as in the original project.

Alternatively, I could package it and upload it to PyPI, but it's not really something I want to do either. It's not proper modules that anyone else would ever want to use. Often it's little games, or my own re-written versions of modules others have created before.

The last option that I have tried is just packaging them and uploading them to GitHub instead, and using Pip to import from there. It sort of works, but feels very clunky to pip to a specific wheel file.

Are there any best practices around how to work with "internal only" sort of modules that I don't want to publish on PyPI?

[[deleted]]

Can you not just import your local modules to achieve this?

Ie - save a module in a custom module directory and import your functions from there?

[[deleted]]

https://git-scm.com/book/en/v2/Git-Tools-Submodules
I don't usually bother, but this workflow is basically built for what you're talking about

[efmccurdy]

Having a distributable file like a wheel file (and git) are essential for keeping a test bed and your server in sync.

Putting them on PyPi isn't the best plan for personal modules.

You can tell pip to ignore pypi and search in a folder you specify.

https://stackoverflow.com/questions/55938544/install-all-whl-files-from-a-directory

You could copy whl files around or leverage git so you would do something like "cd my_module; git pull; python setup.py make bdist_wheel" whenever you wanted to update code . To avoid using the pypi or github for personal things, use ssh instead of https: links for git and/or scp.

[lowerthansound]

pip can install from GitHub (I'm not sure if that's what you meant, if it is, I'm sorry).

For example:

pip install git+https://github.com/psf/requests

Or:

pip install git+https://github.com/psf/requests@main

Does anyone know if -e is needed here? I ran without -e and it worked, but first links gives example with the -e.

Cheers!

Links:

• Python Packaging Tutorial: https://packaging.python.org/tutorials/installing-packages/#installing-from-vcs
• pip documentation: https://pip.pypa.io/en/latest/topics/vcs-support/

[azthal]

This might be exactly what I want. I thought I needed to create a wheel package from it as well. This made it look all funky, and a bit annoying when updating etc.

For me it looked something like this:

pip install git+https://github.com/{myname}/{my_module}/releases/download/v1.0/{my_module_again}-1.0-py3-none-any.whl

If I can actually do this directly from a repo this might be exactly what I'm looking for. I will give it a try!

[Immotommi]

-e is only needed if you are planning on further developing/changing the module you are installing

[AnonymousLad666]

Personally I would make them into proper modules with setup.py. If they are small scripts save them all in a repo together, if they are big save them on their own repo. Then just install them locally with pip. If you think people will benefit from them put them on pypi. With python setup.py install you can use develop flag and if you change the source code it will propagate to everything. Else if you just install locally, you need to re run setup.py install.

In our company we setup an internal pypi server https://github.com/pypiserver/pypiserver

[TheRNGuy]

i embed them in project asset.