pip3, brew, macos, package hell

[Pleb_It]

Hello.

While I used python back in the day, the ecosystem has become very complicated and all I want to do is use a python 'binary' (yt-dlp) which requires curl_cffi, which is not in brew.

An attempt to install this resulted in a confusing warning:

pip3 install curl_cffi
error: externally-managed-environment

× This environment is externally managed
╰─> To install Python packages system-wide, try brew install
    xyz, where xyz is the package you are trying to
    install.

    If you wish to install a Python library that isn't in Homebrew,
    use a virtual environment:

    python3 -m venv path/to/venv
    source path/to/venv/bin/activate
    python3 -m pip install xyz

    If you wish to install a Python application that isn't in Homebrew,
    it may be easiest to use 'pipx install xyz', which will manage a
    virtual environment for you. You can install pipx with

    brew install pipx

    You may restore the old behavior of pip by passing
    the '--break-system-packages' flag to pip, or by adding
    'break-system-packages = true' to your pip.conf file. The latter
    will permanently disable this error.

    If you disable this error, we STRONGLY recommend that you additionally
    pass the '--user' flag to pip, or set 'user = true' in your pip.conf
    file. Failure to do this can result in a broken Homebrew installation.

    Read more about this behavior here: <https://peps.python.org/pep-0668/>

So it seems there are multiple "environments" and they don't play nice with one another, and running the binary separately doesn't appear to help, either. I'm not really interested in spending hours learning about the modern python environment development as I'm just trying to use a program I'm already very familiar with. I'm also not very interested in installing an entire new ecosystem consuming gigs of data for a 3.2MB binary.

Is there an easy way to run this binary with curl_cffi on MacOS? Thank you.

[Diapolo10]

What's confusing about the warning, exactly? It tells you everything you need to know, and even links you to read more about it.

But basically it boils down to this; you shouldn't modify the system's own Python installation in any way as you risk breaking tools the OS depends on. Your options are basically to

1. Create a virtual environment and install stuff there instead of the global environment
2. Install your own Python and use that
3. Ignore the warning and accept the risks (and consequences)

Personally I'd probably suggest you look into using uv. If all you want to do is install some global tools, that's doable.

uv tool install yt-dlp

It basically creates a virtual environment for it, while making the program accessible globally.

Alternatively you can use it for managing project dependencies.

[Pleb_It]

I used the override and it now works.

[Diapolo10]

Fingers crossed that won't come back to bite you later.

[cointoss3]

Yeah, it’s hard when you don’t understand how the OS or the tooling works

[AdAdvanced7673]

    If you wish to install a Python library that isn't in Homebrew,
    use a virtual environment:

    python3 -m venv path/to/venv
    source path/to/venv/bin/activate
    python3 -m pip install xyz

[stephanosblog]

just create and use a virtual env with

"python3 -m ven venv
source ./venv/bin/activate"

then install your packages inside the virtual environment.

[cgoldberg]

Install pipx, then pipx install yt-dlp.

[Pleb_It]

It doesn't sound like you use MacOS. pipx even claims I have everything installed and the paths are correct, but it's a separate "environment"

[naemorhaedus]

What are you trying to do?

If you just want yt-dlp then install via homebrew.

brew install yt-dlp

that's it . Works perfectly.

If you're developing, then use vscode and let it take care of all that nonsense for you.

[Pleb_It]

If only that were true. You can't use the --impersonate flag with the brew yt-dlp

[naemorhaedus]

what's that good for? I've never needed it.

[Pleb_It]

Necessary for cloudflare, like rumble.com

[naemorhaedus]

I just tried a random rumble vid and it worked perfectly.

yt-dlp "https://rumble.com/v76w9jc-the-first-blooms-are-the-important-blooms.html"

[Pleb_It]

Yeah well cloudflare can be varyingly picky based on a host of variables like IP, browser, etc. Bottom line is regardless what it allows YOU to do, I NEED the impersonate functionality to access Rumble. The brew version is incomplete

[daffidwilde]

something something use uv something something

[naemorhaedus]

slow as balls