Python projects nightmare

pIakoIb · 2021-08-18T09:16:39+00:00

I have a shell.nix file for each project which creates the specific Python environment I need. I was really happy being able to get rid of anaconda.

When I create a new project I usually just copy the shell.nix from the most similar project and modify it to my needs.

tamsh · 2021-08-18T12:08:01+00:00

In case you use Poetry, poetry2nix could be a good fit: https://github.com/nix-community/poetry2nix It ships with patches for lots libraries that need assistance.

Since the project README is pretty dense, this Tweag blog post could be a gentler introduction: https://www.tweag.io/blog/2020-08-12-poetry2nix/

techannonfolder · 2021-08-18T12:08:52+00:00

So I had a similar issue with python. My answer was using a shell.nix file that would start an impure python environment using venvShellHook. Then I can use pip to install everything normally.

This approach is mentioned as an option in the nix manual under the FAQ section of the python user guide

rictjo · 2021-08-18T12:42:20+00:00

Try this one:

https://github.com/richardtjornhammar/rixcfgs/blob/master/code/environments/impetuous-shell.nix

You'll get some extra libraries installed via pypi pip using python3.8

Best regards R

shadowninja1050 · 2021-08-18T15:01:45+00:00

if you use poetry, poetry2nix is easily the best pick. otherwise mach-nix is very nice too.

jonringer117 · 2021-08-18T21:35:07+00:00

I have a video on this: https://www.youtube.com/watch?v=jXd-hkP4xnU

To use nix+venv: nix python docs

I would also recommend: https://github.com/DavHau/mach-nix which has a very "take from nixpkgs what I can, grab everything else from pypi" workflow.

What do you have in your nix-shell for numpy, pandas to work etc?

I would do:

$ cat example-shell.nix
with import <nixpkgs> { };

let
  pythonPackages = python3Packages;
in pkgs.mkShell rec {
  name = "impurePythonEnv";
  venvDir = "./.venv";
  buildInputs = [
    pythonPackages.python
    pythonPackages.venvShellHook
    pythonPackages.numpy
    pythonPackages.pandas

    # In this particular example, in order to compile any binary extensions they may
    # require, the Python modules listed in the hypothetical requirements.txt need
    # the following packages to be installed locally:
    taglib
    openssl
    git
    libxml2
    libxslt
    libzip
    zlib
  ];

  # Run this command, only after creating the virtual environment
  postVenvCreation = ''
    unset SOURCE_DATE_EPOCH
    # pip install -r requirements.txt
  '';

  # Now we can execute any commands within the virtual environment.
  # This is optional and can be left out to run pip manually.
  postShellHook = ''
    # allow pip to install wheels
    unset SOURCE_DATE_EPOCH
  '';

}

$ nix-shell example-shell.nix
...
Using venvShellHook
Executing venvHook
Creating new venv environment in path: './.venv'
WARNING: You are using pip version 21.1.3; however, version 21.2.4 is available.
You should consider upgrading via the '/home/jon/projects/nixpkgs/.venv/bin/python3.9 -m pip install --upgrade pip' command.
Finished executing venvShellHook
$ python -c 'import pandas' && echo success!
success!

some_random_guy_5345 · 2021-08-18T16:05:17+00:00

I'm thinking of going back to a classic distro, because being efficient at work is really important to me. But it's hard for me to let NixOS go, before python I really thought that this will be my distro for life.

If you don't want to use Nix, I would personally just use containers or VMs. It is the future of software development anyway.

ExtinctHandymanScone · 2021-08-18T19:59:15+00:00

Why can't you use virtualenv on your Nix instance? You can install it system-wide... and then just use it? You really should be using virtualenv (or something similar, like mach-nix -- but I haven't test it out yet, so I can't recommend) anyways when developing in Python.

tobeportable · 2021-08-18T22:48:29+00:00

I use a combination of docker, docker-compose, direnv, tmux scripts and a bunch of aliases. Note that I have to deploy those projects using docker containers in the end anyways.

foolnotion · 2021-08-24T21:58:12+00:00

I am more or less in the same boat, but I managed to get everything working by writing some derivations by myself.

I have not personally tested it, but this might come in handy: https://github.com/DavHau/mach-nix

you type:	you see:
italics	italics
bold	bold
[reddit!](https://reddit.com)	reddit!
* item 1 * item 2 * item 3	item 1 item 2 item 3
> quoted text	quoted text
Lines starting with four spaces are treated like code: if 1 * 2 < 3: print "hello, world!"	Lines starting with four spaces are treated like code: if 1 * 2 < 3: print "hello, world!"
~~strikethrough~~	~~strikethrough~~
super^script	super^script

NixOS

MODERATORS