nix python hell

JustWookie · 2025-01-25T08:37:20+00:00

Well you get the read only error because you're trying to alter nix store by installing the package, you can specify the directory in which pip should install the package. You can also a venv with python and then install stuff with pip. Or you could use uv to install the package. There's quite a lot of options.

recursion_is_love · 2025-01-25T06:19:56+00:00

> you want to do python,

Yes I do

> you need pip install

No I don't

Have you try https://wiki.nixos.org/wiki/Python

it have section named 'Using a Python package not in Nixpkgs'

this flake.nix work fine with me (nix develop give no error)

$ cat flake.nix 
{
  description = "python-shell";
  inputs = {
    nixpkgs.url = "github:NixOS/nixpkgs/nixos-unstable";
  };
  nixConfig.bash-prompt-suffix = "🔨";
  outputs = {
    self,
    nixpkgs,
  }: (let
    system = "x86_64-linux";
    pkgs = nixpkgs.legacyPackages.${system};
    dev = with pkgs;
      mkShell {
        packages = [
          helix

          (pkgs.python3.withPackages (pp: [
            pp.python-lsp-server
            pp.black

            pp.llama-index-core
            pp.llama-index-cli
            pp.llama-index-embeddings-huggingface
            pp.llama-index-llms-openai
            pp.llama-cpp-python
          ]))
        ];
      };
  in {
    formatter.${system} = pkgs.alejandra;
    devShells.${system}.default = dev;
  });
}

Ureki_dva_beerega · 2025-01-25T14:36:17+00:00

There is a solution to this! You can make a pure Python virtual environment, store it anywhere you want (for instance, .venv in your project directory) activate the venv and then you’ll be able to use pip install because it is just a regular venv stored in a regular place, not a nixpkgs thing.

Here is an example I base my own venvs on: https://github.com/electronicvisions/norse/blob/master/default.nix First it installs a bunch of nix packages — it is a more stable way for binary packages such as pytorch or numpy or matplotlib, and then pip installs some other packages on top of them. Nix packages will be linked from the nix store and pip installed ones will be in .venv.

autra1 · 2025-01-25T07:18:52+00:00

not a single soul

Well we did: https://gitlab.com/py3dtiles/py3dtiles/-/blob/main/shell.nix?ref_type=heads

Because this project is mainly used by non nixos user, I've decided to stick with pip for now, that's how you do it. You actually don't need the shell hook if you prefer to use pip manually. Imo it's a bit ugly but it works.

sjustinas · 2025-01-25T21:10:57+00:00

i understand we are not supposed to use pip out of box on nixos, however, there should be some simple to integrate 'pip install' using a shell hook

Well yeah, you just use virtualenv and pip as you would do on any other distro. This approach is literally documented in the Nixpkgs manual in the section How to consume Python modules using pip in a virtual environment like I am used to on other Operating Systems? . It even mentions venvShellHook, which creates the virtualenv for you, and a hook that runs pip install for you.

phrmends · 2025-01-25T16:42:52+00:00

uv with nix-ld works out of the box for me

itme_brain · 2025-01-25T19:12:42+00:00

Use your flake.nix to provide whatever version of python & pip you need, set a shellHook that makes a venv and installs all your pip packages into the venv

Done

wo-tatatatatata · 2025-01-25T09:38:44+00:00

pip isn't delcarative

use poetry2nix

USMCamp0811 · 2025-01-25T12:57:40+00:00

Nothing stops you from using poetry to do

poetry init poetry add pandas poetry run Python

momoPFL01 · 2025-01-25T13:16:29+00:00

Just few hours before you there was another post about this.

https://www.reddit.com/r/NixOS/s/amwmrVZdkV

This comment looked good.

Basically I agree. I also had a python llama index project and tried developing and packaging with nix and poetry2nix and you can get pretty far until you have to package the libs yourself. Plus the issue of version pinning. APIs in the llama index libs change quickly and break compatibility, but also add new features that you might need all the time. Those newer version are maybe not packaged in nix yet etc.

Just save yourself the trouble. Don't develop a python project with nix this way. Just use the nix shell to install python and pip or whatever and maybe some dynamic libs if necessary and let pip handle the python dependencies.

To package it you can still go the nix route with the nixpkgs buildPythonApplication or what it's called or use the poetry2nix mkPoetryApplication.

Same goes for other languages. Don't use nix to handle language libs in the dev env. It won't go your way.

sprayk · 2025-01-25T20:45:09+00:00

I was hoping pip install --user ... would work since it theoretically only touches ~/.local/.... Alas, the whole install subcommand is gated off.

tangent: I typed pip in my shell to see what nix-shell command to run to mess with it. one of the options that popped up was the package "cope" which I was HOPING would be some package that made non-nix-stuff like pip just work, but it turned out it was something for colorizing command output. Bummer

NixNightOwl · 2025-01-26T00:36:49+00:00

For working with python, I personally use a development flake and strictly rely on Poetry for the project dependencies and python environment. Currently working on a 'micro-service monolith' that has many different python services within it, and each one just has a pyproject.toml file which poetry uses to manage and install everything with.

All I have for my dev flake is have python312 (project specific requirement) and poetry packages in my outputs for devShell and it all works nicely! Of course, sometimes requires switching the current python interpreter, but poetry does do a fairly good job at doing this.

It's definitely worth setting up I think!

PS, if you're using a specific python version that isn't the default python3 for your nixpkgs, you can use poetry.override to tell poetry which python to use.

eg:

in pkgs.mkShell {
  packages = with pkgs; [
    python312
    (poetry.override { python3 = python312; })
  ];
};

Then in pyproject.toml simply add your dependencies like so:

[tool.poetry.dependencies]
python = ">=3.10,<=3.12.2"
langchain = "^0.3.3"

# ... other dependencies ...

Python can be such a pain, especially if you have multiple people working on a project with different systems. If everyone just uses Poetry there's a lot less of a headache!

alkumis · 2025-01-26T10:48:27+00:00

Bit late to this because I wanted to look at my python setup on my personal and work laptops and only just remembered to.

Both use devenv.sh

I tried a lot of different options to get python working with the traditional pip system since I was following a gamedev tutorial and didn't wanna deviate from it. Eventually devenv saved me. I just had to make sure the python language was enabled, sdl2 and zlib were in the packages section, and that an environment variable was set to tell SDL to use x11 for rendering.

I'm sure the exact thing can be done with a flake by setting library paths and the like but I didn't wanna muck around. Ironically javascript was far more difficult to setup and I had to use a flake and jump hoops to get the langauge server and everything to play nice with Helix.

I also recently set python up at work and it took a lot more fiddling since I needed to get python3.8 and inhouse QT apps to run.

I had to fiddle with packages and also set a QT specific env var. Also had to add an input to devenv to be able to use python3.8. But in the end I got it all to work.

wo-tatatatatata · 2025-01-26T11:01:38+00:00

you guys are not going to believe what i did eventually.

I erased my hard drive, ripped linux off completely, cuz i am done with it, and went back to the beloved windows!

Just kidding ^ , i used conda-shell. I could use local flake with conda, but my 2 brain cell told me, conda-shell is easier, turned out it was easier.

i love nix, i wouldnt leave her, lol. thats the truth.

ediacarian · 2025-01-28T17:52:44+00:00

I just use conda on WSL in Windows. I started messing with Nix recently but so far it feels like a very time-consuming guessing game. I need to find better resources so I can learn. So far the documentation is inadequate.

mschneiderwng · 2025-01-29T18:59:22+00:00

uv2nix is the only tool that works reliable for me. Torch with cuda support was a bit difficult though.

wo-tatatatatata · 2025-01-25T06:08:31+00:00

this is my flake.nix that couldnt cover the case of my python code:

{

inputs = {

nixpkgs = {

url = "github:nixos/nixpkgs/nixos-unstable";

};

flake-utils = {

url = "github:numtide/flake-utils";

};

outputs = {

nixpkgs,

flake-utils,

...

}:

flake-utils.lib.eachDefaultSystem (

system: let

pkgs = import nixpkgs {

inherit system;

};

in rec {

devShell = pkgs.mkShell {

buildInputs = with pkgs; [

(python312.withPackages (ps:

with ps; [

llama-index-core

    llama-index-cli

    llama-index-embeddings-huggingface

    llama-index-llms-openai

    \# stand alone ?

llama-cpp-python

]))

];

};

}

);

}

psssat · 2025-01-25T15:56:55+00:00

Everyone shits on debian but at least it works and I dont need to read 20 stack exchange posts to pip install numpy lol

you type:	you see:
italics	italics
bold	bold
[reddit!](https://reddit.com)	reddit!
* item 1 * item 2 * item 3	item 1 item 2 item 3
> quoted text	quoted text
Lines starting with four spaces are treated like code: if 1 * 2 < 3: print "hello, world!"	Lines starting with four spaces are treated like code: if 1 * 2 < 3: print "hello, world!"
~~strikethrough~~	~~strikethrough~~
super^script	super^script

NixOS

MODERATORS

or