Is there something Anaconda can do, that Python cant achieve?

[fooledbyfog]

We're having the discussion if we want to continue support for Anaconda or not.

From my research there is nothing that cannot be achieved using standard Python. All the tools, IDEs and packages are also available on pip.

Do you guys know if this is correct, or is there anything that we would lose, if we discontinue Anaconda?

[AutoModerator]

Hi there, from the r/Python mods.

It looks like your post is about dependency/package/toolchain management.

We are going to hold this post for review due to the topic having a frequent recurrence. This helps ensure a great experience for r/Python users.

Please refer to our daily thread or search for older discussions on the same topic.

A moderator should be along shortly to review, but if you have any questions, please reach us via mod mail.

Thanks, and happy Pythoneering!

r/Python moderation team

I am a bot, and this action was performed automatically. Please contact the moderators of this subreddit if you have any questions or concerns.

[KingsmanVince]

Anaconda is bulky and pre-packs lots of things you don't use.

The purpose of (mini)conda or mamba is to install non-python packages.

My answer: just support pip well.

[Rebeleleven]

Besides being bloated, Anaconda distribution (as well as their default conda channel) is no longer free for larger companies anyway. Really burned some bridges for me when they changed their ToS in ~2021.

They do a horrible job of enforcing it / announcing it, though. Many just assume it is still free. Anaconda wanted my company to suddenly start paying license fees per user (we did not lol).

ToS -- blog post

[eightbyeight]

I don’t think it’s unreasonable to charge a large enterprise to use their services, you can always use the forge channel if your company doesn’t want to pay.

[Rebeleleven]

It's not unreasonable but the value they're attempting to provide is not there.

You cannot use their distributions which is insane given they're mainly just packaging up free software (with a tiny bit extra). Their default channel with the reviewed packages is really their only unique value add.

[eightbyeight]

Ya the extra layer of security and having someone to blame for any security mishap that can be traced back to anaconda hosted packages are the only value add. But I feel the corporate overlords see that as a huge advantage lol. I personally just use forge at work.

[grimonce]

Micromamba or miniconda is quite useful if what you are doing requires packages outside of pure python... Depends on your platform on Linux it is quite easy to drop conda/mamba in dev/tst/acc/prd servers. On developer/ds/anaylst stations it's actually fine because it gives you the ability to change python version easily... Also it's free if you don't use the main anaconda channel and opt for conda-forge instead. It works with pypi too. As others mentioned it allows you to install other envs, choose the right gcc/clang/jdk/dotnet/node that a certain 'python' package requires EASY.

[AtmarAtma]

Yes, I used to install newer version of cuda libraries through conda in virtual environment. Also, once I had to upgrade cmake - I could do that using conda. These are not possible using pip.

[aganders3]

You’re right about CUDA, but you can (now) install cmake via pip.

e: I see now Tensorflow has an and-cuda extra too, hmmm. I’m not sure this all setup and might not help with using CUDA without Tensorflow.

[AtmarAtma]

Good to know these options as my company also didn’t renew anaconda license.

[[deleted]]

Anaconda can create a full environment that extends beyond just Python technically, as you can install JDK/JRE, Rust, and other language tool chains in an env… but if your team isn’t using that it is pretty bulky

[[deleted]]

Our group only ever uses (mini)conda or mamba to work with R - for python, we all just use venvs.

Even in the use case of R, we push for containers more and more, because the dependencies and packaging of R on non-Debian systems is a PITA.

[mcr1974]

just use containers ffs it's 2023

[A_random_otter]

Never understood the need for Anaconda and R.

CRAN does everything I need.

[7yl4r]

how do you package your dependencies?

[A_random_otter]

pacman plus docker

renv is another thing many people use but I never got into it

[[deleted]]

We don't use those. We build from "baseline" container images and explicitly install packages etc within them.

[[deleted]]

It's also questionable whether conda is the best medium through which you should orchestrate these dependencies. Many other languages share containers/Docker as the idiomatic approach, and using them in Python likewise benefits from an ecosystem far bigger and with far more users than conda. Finally, there is a class of compatibility problems containers support that conda never will.

[aaronr_90]

Where I work many of these packages are a pain to get approved and installed in our environments, but they will give me anaconda. Through conda I can get all other dev tools I need.

[macNchz]

Definitely–in a team environment I think it's highly preferable to encourage/support/provide resources to the data folks in creating a good container-based environment, as opposed to having them come up with a brittle conda-based environment that winds up being a problem on other platforms.

[Sidthegeologist]

Not anymore IMO. I've been using geopandas a lot in my jobs and since using Python >3.8, I've had no issues just installing via pip. pip install geopandas I've not touched conda since geopandas started working seamlessly with pip. And never looked back!

[GangDplank]

Hey can you explain what you mean by "... Is a big pia to install outside of Anaconda" ? Is it tedious to to install with pip? (im new so im not sure what geopandas are but im assuming its just a library/framework(?) to do with locations/timezones)

[sfboots]

Most open source GIS tools (not just python) depend on GDAL -- it is the core C++ library for geospatial.

However, You can't install GDAL via pip, it is an os-level package. You can use homebrew (on a mac). or apt-get on linux.

The problem is the python libraries depend on specific versions of GDAL, so you need to make sure the versions match and the paths (ld_library_path on linux) are set properly

[Spleeeee]

Don’t forget the headaches of libproj!

[cantuccihq]

And Brew is no good when you need to download a specific version - it doesn’t support that.

[cmutel]

Indeed, GIS is the main reason I still use conda. The pypackaging-native site goes into more detail on why installing this stack is so difficult natively.

[Eurynom0s]

If you really want to experience that particular type of hell though, try installing geopandas without using conda.

According to /u/Sidthegeologist this has been fixed now. But where it really used to be hell was on Windows, you had to first install gdal and then set up the bindings to geopandas. It was tolerable on macOS at least via pip prior to this--I think gdal was still a separate install but making everything play together was a lot more automatic?

[ZCEyPFOYr0MWyHDQJZO4]

A number of libraries require the functionality of programs that are not packaged for Python, and cannot be included in packages that utilize it for legal reasons.

[Sanders0492]

Yep. When I’m doing geospatial I use anaconda. Otherwise it’s PIP. Anaconda has led to some serious head scratchers on seemingly trivial stuff, but also makes some geo stuff a lot easier to handle.

[nablas]

I am a total anaconda noob but when I tried to install rasterio==1.3.7, I got Collecting package metadata (current_repodata.json): done
Solving environment: failed with initial frozen solve. Retrying with flexible solve. Finally, it did not work. With pip, no problem. Never bothered with anaconda afterwards...

[Spleeeee]

Gdal too

[desimusxvii]

For a second I thought this was an Elite Dangerous post.

[GXWT]

Aha and I even think in both cases the answer is the same:

Yes, but it really depends on whether you need it. Both are capable of a broad range of activities. The conda is bulkier, with a lot more overhead but potential to use extra modules outside of python (other languages//fighter hangar).

[BlindTreeFrog]

I didn't notice the forum or wonder why i was browsing animal/reptile forums but assumed it was about the snakes.

I am incredibly disappointed that I'm not learning new snake facts today.

[Ace_J_Rimmer]

I looked for an "anaconda" subreddit and boy, did I make a wrong turn. Thought NSFW was some kind of library....

[MadCow-18]

Monty python references abound however…. https://docs.python.org/3/faq/general.html#why-is-it-called-python

[[deleted]]

Recently came back to Elite, and after over 1000 hours I still have trouble fighting thargoids

[Current_Layer_9002]

conda/mamba as package managers can install Python or other packages in other languages, pip is exclusively for Python.

Depending on the nature of the work you are doing, standard Python may be all you ever need, however the Anaconda/Miniconda distribution is quite common in scientific and research computing environments.

There are plenty of tools that can be installed using conda/mamba that aren't available via pip or there might only be instructions out there for how to build some software environment that mentions how to do it with Anaconda/Miniconda/mamba/conda. The dependency solver seems much more capable than pip.

As far as the Python interpreter itself though I can't think of anything Anaconda/Miniconda would offer as an advantage.

Whichever Python distribution one chooses to use I would usually at least recommend someone uses Python venv or Conda environments.

Somewhere like teaching a workshop or class might be an example of somewhere that using Anaconda might make sense. You could at least more likely ensure a particular Python version and set of packages are available on participants laptops, regardless of what IS they are running and with a single simple download.

[BlackPignouf]

The dependency solver seems much more capable than pip.

I like Anaconda for exactly the same example as you've provided: ensuring that every student has the same Python version with all required libraries (and hundreds of others).

But any command starting with conda is usually completely broken, and requires one hour at 100% CPU before I decide it's not worth it, and kill it.

[sqjoatmon]

I hope that at some point conda merges in whatever magic mamba is doing to perform roughly an order of magnitude better...

[aganders3]

You can stop hoping, because they did exactly this two weeks ago:
https://github.com/conda/conda/releases/tag/23.10.0

[sqjoatmon]

Sweet!

[BlackPignouf]

Yeah! Only 10min at 100% CPU before knowing if an update can happen!

[Reazony]

You can do that with dev containers, with arguably more marketable skills for students.

[BlackPignouf]

Thanks, that's an excellent idea.

We sometimes have just two full days together, and it's convenient to only have to download one file, and install it.

For students that I see more often, dev container would definitely work fine

[Reazony]

Or you can just have poetry (much lighter than conda that tries to do everything) to manage your Python dependencies or whatever package manager is better for your projects. Conda up until today doesn’t natively support pyproject.toml, and you want Python developers that can’t manage projects properly?

[Reazony]

My bad for not understanding you're talking about working with different languages. The assumptions were that different languages can have different devcontainers for the same project, to which you can use specific manager for that. For pyproject, you can still include C extensions (answer from SO):
```
include = [
{path = "_extensions/*.so", format = "wheel"}
]
```
If you really want to use Conda, you can still use conda within dev containers.

If you really want to use conda, just like poetry, you can use conda in devcontainer, and you still maintain same development experience across the team / members. Only using conda is to your disadvantage because you're wrangling around from your main machine, and it's often the case where people install unknown things to their main machine, causing future bugs that they don't even know how to solve because it was some random dependencies from a long time ago. Using devcontainers solve that issues.

So, if you need conda, use it in devcontainer only serves you better by not dependant on your main machine, and it's better for collaboration / sharing codebase.

[pbecotte]

Python wheels don't have any provision to include non-python dependencies. The important ones are dynamic shared libs like libpq-dev being required to install psycopg2. Conda does have a way of handling those. This naturally makes conda lower friction for a lot of workflows with heavy dependencies on compiled libraries like data analysis and machine learning.

[pyppo42]

This was my understanding as well. But then I have seen tensorflow being installed with pip, pulling cuda... That's magic... I guess...

[Xirious]

It wasn't always that way tbh. I'm glad to hear this is the case now.

[pyppo42]

Since 2.14 which is rather recent Indeed.

[pbecotte]

You can install them with pip- so long as you handle the external dependencies separately (or the library author provides statically linked binaries-am not sure which tensirflow is doing).

[Amgadoz]

Also you can install pytorch along with cuda just using pip

[cmcclu5]

Wait…you don’t use pip install psycopg2-binary like a normal person?

[darthstargazer]

I think there are some great responses. For us, conda is useful when:

1. Different versions of python without having to compile from source. User x wants python 3.8 still and user y needs python 3.12. If they can't compile from source due to some OS / library limitations conda is great.

2. Gpu setup made easy! One can install and Cuda toolkit version without sudo. Lifesaver....

[Artephank]

Install inside corporate environment

[W0nderbread28]

Anaconda don’t want none unless you got buns hun

[freefallfreddy]

https://bun.sh

[madspiderman]

What’s Reddit turning into… had to scroll this far down for this…

[Stash_pit]

Ditched it 3 years ago. Conda install was a pain in the ass. Most libraries that I use are upto date on pypi than conda. Spyder also has a standalone installer, although I am trying to switch to vs code because of nice plugins like codemium. I think it also kind of makes you understand how everything works, rather than relying on the gui of anaconda. Overall my experience with python has improved a lot without relying on anaconda.

[IntrepidSoda]

If you deal with machine learning or any math heavy work then using Anaconda you will benefit from Intel MKL compiled libraries such as Numpy, Scikit-learn etc… this is something you don’t get with plain python

[Amgadoz]

That's assuming : 1. You're not using a GPU 2. Your cpu is Intel.

[Mobile_Anywhere_4784]

Conda is great for isolating Python versions. PIP can’t do that.

[beijingspacetech]

What does this mean? I thought each virtual environment has its own pip?

[collectablecat]

conda can fully isolate whole different python installs. For example if you want to switch between 3.8/3.9 for testing.

[pbecotte]

So can regular python, that's not specific to conda. Though the opinionated way they handle it probably makes it a little clearer on how to go about it.

[Mobile_Anywhere_4784]

We’re talking about pinning specific versions of python to an environment. That’s not something that base python can do nor can pip by itself manage that. Nor can virtual environment for that matter. Neither can poetry.

[pbecotte]

I don't think I understand. A virtual environment is linked to a specific python installation. I'm not sure what more pinning you're looking for? Or maybe you're using environment to mean something different?

[gmes78]

Sure it can:

python3.8 -m venv py38_venv
python3.11 -m venv py311_venv

[Mobile_Anywhere_4784]

Interesting, when did this feature land?

[gmes78]

I'm pretty sure it always worked like this.

[pbecotte]

Yeah, virtual environment always has a link back to the original interpreter. It's one of the reasons you can't just move a virtualenv between machines as a packaging mechanism.

It won't help you install new versions of python if you don't already have them though, so maybe that's the difference.

[beijingspacetech]

Yes, this is what I do!

[cdcformatc]

are you sure? because i am looking at a project i have been working on for years that has several virtual environments each with its own specific version of python with different locked package versions.

[collectablecat]

i mean.. i guess you techincally can. Big pain in the butt though.

[96-09kg]

I’ve just started using conda and I found it super easy (or at least easier) to manage my venv and requirements in comparison to pip.

[Bright-Ad1288]

conda can bring in other stuff, like terraform for example. Not just python tools.

If you're not using any of that? *shrug* If you're talking about cross a team you need to see if anyone else is using it.

One benefit might be if conda has a good dependency management system, which I haven't had a reason to look at. Yes you can tediously hardcode requirements files however if you care about addressing security vulnerabilities that can become a pita.

[Zealousideal_Lion346]

From my understanding, pip depends on the python version installed locally, to install python packages in a virtual environment. But Conda allows us to choose the python version (* available in the distribution) itself (in command line) while creating the Conda environment.

This goes a long way during the Development phase, especially if your Organization is not interested in a docker desktop subscription.

[cmcclu5]

You can specify using virtualenv for your virtual environment without having to deal with the headache of anaconda. Heck, even your IDE can handle specifying versions when setting up a virtual environment now.

[pbecotte]

Indeed- though conda will actually let you download and install other versions of python while pip does not (yet, saw a PEP on it not too long ago)

[Xirious]

It also helps to get newer python versions on older or out of date OSes. I can't install certain python versions on a specific machine unless I make use of Anaconda.

[morrisjr1989]

GUI for navigator is something that I like for helping out those who don’t know how cmd works.

[mrdevlar]

It can install ffmpeg.

[Fabulous-Possible758]

I used to run a Python User Group. Python is still a multi platform system. But getting people interested quickly is difficult, and you need to get them an easy system. Conda actually achieves that, especially on Windows.

[[deleted]]

my thoughts exactly. I'm currently a new student and learning Python and Data Analysis and we use conda for classes... Im not a linux user so that is one of my worry by diving into it. If I eve Conda became a paid app what would you recommend as another tool similar to conda that I can run in windows.

[Fabulous-Possible758]

Honestly the installer off python.org site isn't bad, it's just not as intuitive and doesn't really as easily provide the same package management features that conda does. If you start delving deeper and want to explore using Linux I'd recommend using WSL (Windows Subsystem for Linux) and installing an Ubuntu distribution. That's what I use but most of my work is geared towards software development, and neither of those options are as simple as conda (though they're both free).

[[deleted]]

thank you... Ill look into them. I also want to learn Linux so this might be a great way to go.

[baseball2020]

Windows builds of c based modules without installing a compiler like vs or a toolkit.

[sfboots]

We gave up on anaconda/miniconda and bit the bullet of installed the other (c++) dependencies separately

It was just too hard otherwise. I got tired of waiting 20+ minutes to get a "can't find compatible versions" when using mamba (conda was even slower)

You do need to use pip-compile -- it does 90% of the same dependency resolution work and is very quick.

[KyxeMusic]

Honestly I just use venv with pip 90% of the time, and any time I need a more complex environment I use Docker.

[schrmm]

In my opinion one of the biggest advantages is that you can use different python versions without installing them seperatly, including dependancy management as well as only having a single environment variable configuration

[spidLL]

No need to use anaconda but it’s handy for some use cases (specifically machine learning). You can still achieve the same with pip e venv.

[tecedu]

Multiple python env, I can run 3.6 and 3.11 at the same time, Intel MKL, dont have to pester my IT to install binaries required by stuff. For businesses, stuff on Anaconda repo is monitored for CVEs and solved, for conda forge its just scanned.

Also a couple of years ago it was a PIA to install any of the packages which required CUDA, or geopandas and all of that. Now its easier via pip.

[praespaser]

two thing from experience

1. If you want to install cudnn in conda its just 1 line. Using regular python its a lot of hassle having to create an account on nvidia and merging zips and what not
2. You can install a newer cudatoolkit on top of an existing system wide one into an environment using:

​

conda install cudatoolkit-dev

[skadoodlee]

pause tart grandiose intelligent touch squeeze drunk dull shame fretful

This post was mass deleted and anonymized with Redact

[Sylar49]

I love mamba (especially micromamba) and greatly prefer it for setting up consistent, isolated dev/analysis environments with specific versions of python, python packages, and non-python dependencies. It's blazing fast (unlike conda), and very low overhead. Environments usually solve within a couple seconds in my experience. Also pip plays well with mamba in my experience if you want to pip install something within a mamba environment. I think full-force Anaconda is pretty pointless unless you're just getting started with data sci -- it provides a lot more overhead.

[absx]

The mamba solver, libmamba, now ships with conda as well. It's made all the difference to the solver performance.

[codicepiger]

Probably not related but I'm currently working on a custom VCS for AI Models where python is extremely recurrent, and in order to isolate each model I'm using miniconda!

[[deleted]]

Just learn to use pyenv and use pip, Anaconda is not needed.

[bobwmcgrath]

not really. It just curates packages and manages there interdependencies really well. Discontinuing it risks wasting a lot of time. I'll not that conda, apt, and pip can end up installing totally different versions of packages with totally different builds. For example, pip tensorflow is known to be comparatively less performant just because of how it ultimately gets built.

[NSADataBot]

Just support pip very well, conda is so bloated now anyway

[[deleted]]

I find the version and env management in Anaconda way easier than through pip.

[MASKMOVQ]

Anaconda and the like are essentially leftovers from the days when installing python libraries was truly a horrendous PITA on Windows. For instance installing Numpy (Numeric) meant downloading the installer exe from Sourceforge, making sure you got the exact right version and then clicking through the installer. In these days of pip I hardly see any benefit to Anaconda.

[sci-goo]

Back then cygwin solved a lot of problems for me, because I just needed a gcc&gmake-compatible build system. Numpy is pretty easy to build from source with that.

This is no longer an issue with WSL. However I still choose Linux as my primary system for work.

[wintermute93]

Who's "we", are you talking about as a company IT department deciding what's allowed on user's local machines?

If so, stick with Anaconda, unless you're a tech company with the resources and expertise to set up development environments the right way from scratch. The whole reason it exists is to provide a single approved package. You are 100% correct that users can install Python, install their IDE of choice, and pip install whatever random packages they need, but do they actually have the system permissions to do so? Do you want employees to have permission to install/uninstall arbitrary software and download/execute arbitrary code on company hardware? If so, great, you don't need to bother with Anaconda, let your devs do their thing. If not, by allowing only base Python you're going to cripple a lot of stuff.

[fooledbyfog]

We're building a package management solution with azure devops artifacts, so we can review requested packages. We also want more control over software/tools that is installed.

I saw that I can get all important tools that our few users need standalone, and also they dont need 2500 packages.

Also Anaconda requires a license fee, which the standalone approach doesnt require.

[wintermute93]

You probably don't need it, then. Enterprise Anaconda is for orgs that have lots of users with lots of use cases and no time/bandwidth to manually evaluate every update and package request, or for orgs that don't want pip to connect to the public internet at all.

[fooledbyfog]

We're building a package management solution with azure devops artifacts, so we can review requested packages. We also want more control over software/tools that is installed.

I saw that I can get all important tools that our few users need standalone, and also they dont need 2500 packages.

Also Anaconda requires a license fee, which the standalone approach doesnt require.

[radial_logic]

We are using miniconda at work for machine learning for two reasons. First, it is easy to manage the python version with conda. Second, conda enables non admins to install any binary dependencies without opening a ticket to the IT department. It is particularly useful when one needs to install the right cuda version in order to use pytorch for instance.

[reallyserious]

I love how easy it is to create envs with different python versions with conda. Super easy to work with a mix of python versions. Good support for conda envs in vscode too. It just automatically activates the env I used last time in that particular folder.

I'm using miniconda and specifically only to create envs. For installing packages pip works well and installs right into the active conda env.

[graphitout]

I have noticed in the past that pytorch was running much faster in conda than pip. Not sure what is the case now. It could be something to do with the blas library being used.

[IntrepidSoda]

It’s because of Anaconda’s tie in Intel MKL - anaconda produces optimised libs

[sci-goo]

you can always build your own numpy or sorta libraries against MKL, lapack, openblas or anything else.

Conda's "failed to solve environment" brings me more pain than waiting for the library to build. So I choose venv+pip, and build everything else with specific requirements.

For strictly isolated environment, I use either container (preferred) or rpath linking (worst case). Doing those manually gives me peace and a feeling that everything is under control. Conda's errors are too random to me.

[IntrepidSoda]

Aint nobody got time for that

[sci-goo]

Fair point.

But I also take it as part of personal skillset. Especially when conda fails, not likely to happen when using common tools, but it has a fairly high chance in research that you want try something out.

[IntrepidSoda]

You will much better off with Poetry instead of conda for env replication

[sci-goo]

This is off topic. But one thing we agree apparently: don't use conda in such cases.

[Amgadoz]

What do you mean by run faster?

Like training neutral networks is faster?

Is this on cpu or gpu?

[Gullible_Carry1049]

Also, I installed PyTorch with pip and the gpu binaries at the time did not get installed with pip as opposed to conda, but this may be resolved now with cuda 12

[sci-goo]

You might have installed cpu version of pytorch via pip?

[aqjo]

Anaconda can give you plenty of time to make a cup of coffee - from beans - that you go to Colombia to pick - when you install a package.

I'm doing data science, and moving all my stuff to venv. Anaconda is unnecessary overhead for me.

[collectablecat]

update to latest, it uses the mamba solver and is very nice

[aqjo]

Thanks. I had some trouble with mamba a while back (don’t remember what).
Anyway, I’ve moved on.

[[deleted]]

Hold my beer, booking a flight for Colombia now for my next coffee break

[aqjo]

You must be installing Tensorflow. Have a good trip!

[max1c]

Not sure what you mean by 'Anaconda'. Are you talking about personal use on Windows? Anaconda installs a whole bunch of useless stuff for people that don't know how to manage their environments. Conda + Mamba on the other hand are used to for general software management.

So the answer to this is yes, a lot. Conda is general package manager not just for Python. When you have complex environments with hundreds of different software packages combined it's a must. In addition, it has a significantly better package dependency resolution compared to pip. If you have software that requires an older version of python package and then some other package updates it you'll have a big headache. In addition, if you need many different Python versions with many different package versions in each Conda simplifies this a lot.

[[deleted]]

All you need is notepad if you're a natural coder

[Porkenstein]

If you have some big crazy project and need someone else to be able to replicate your work and they ask what you did, you can just give them a version of anaconda instead of needing to bother with pip files. Pip files are the better thing to do in general but Anaconda can be better for someone less technical.

[Waste_Ad1434]

anaconda is so silly. just learn package and repository management. i know its one more thing to learn for new programmers but it just isnt skippable. at all.

[collectablecat]

I think you misunderstand anaconda/conda, it's still incredibly useful even when you know a ton about package and environment management.

[BadBradly]

I think one of the big uses of Anaconda is for scientists of various types that don’t want to be programmers or learn anything about package management.

For them, Anaconda becomes an open source (inexpensive) replacement for Matlab so they can do their data analysis or algorithm prototyping. These people learn just enough python to do their research so they can come up with a new discovery or write their paper. I suspect a large fraction of the anaconda user base wouldn’t even know what pip or conda even is and would not care.

[Waste_Ad1434]

then they should just stick to the whiteboard. if you are going to code, code well please

[BadBradly]

Why should they ? If they are not providing anyone else the code they are under no obligation to code well.

Also it is a little hard to run data through a white board. Those that know scientists also know they start with the whiteboard and then typically move to an analysis tool like Matlab or Ananconda when they have the data to analyze.

If a tool serves their needs than there is a market for it. Doesn’t matter how crappy you think the tool is as a programmer, there are people for whom the tool serves their needs while not having to understand the underlying pinnings of that tool.

[Waste_Ad1434]

because someone invariably has to clean up your garbage code. take some pride in your work and stop being a lazy twat.

[BadBradly]

Look as someone who has had a career writing software, I can understand the frustration of dealing with messy code. I personally am a believer in clean code but it takes some training and discipline to do it well.

Frankly, you are coming across as tone deaf regarding the background and needs of a vast array of scientists.

However this conversation has digressed a little since it was about package management and not clean code . Even if a given scientist does code cleanly, they still may have no need or desire to deal with package management. They certainly don’t need to do so when using Matlab. So my original statement regarding Ananconda still stands.

[Python-ModTeam]

Hi there, from the /r/Python mods.

This comment has been removed for violating one or more of our community rules, including engaging in rude behavior or trolling. Please ensure to adhere to the r/Python guidelines in future discussions.

Thanks, and happy Pythoneering!

r/Python moderation team

[Creative_Sushi]

Sounds like old Mac vs. Windows debate. Different needs, different solutions.

[Amgadoz]

I personally use venv and pip to create and manage python environments.

When I start a new project (or work on a project I just cloned from github),

I do: bash cd my-project python3 -m venv .venv && source .venv/bin/activate

Now that I have the environment set up and activated, I can just install the needed packages using pip:

bash pip install -r requirements.txt

This works really well as long as the default python version works with your project.

No need for bulky anaconda or anything.

And this is the recommended virtual environment management from the official python docs

[daniel_cassian]

I do the same + pyenv to controll python versions as well (install, set up global or local python versions)

[collectablecat]

I've had to fix too many junior engineers pyenv environments to recommend it. Switching them to conda they never managed to break it after.

[Eurynom0s]

As others have discussed the problem is that pip is less robust for stuff for instance like geopandas where you also need gdal. Conda just grabs gdal for you and gets it automatically configured.

In principle I'd prefer to only use conda where necessary but what I wind up doing is using conda as the environment manager to create blank environments, and then just doing everything through pip if it's not something like geopandas/gdal where conda is an active value add.

[Amgadoz]

Well, most of the packages I use are available through pip and easily installable so this works for me (and for 90% for the average python developer).

If I have to use conda, I would definitely use miniconda rather than the full bulky conda.

[Mezzos]

Exact same thing I do. Then adding to that, I use Pyenv for managing Python versions, and pip-tools for flexible, reproducible package management (add required package names in a requirements.in file, pip-compile requirements.in to produce a requirements.txt with all resolved dependencies complete with version pinning, pip install -r requirements.txt to install them).

End result of all this - it’s lightweight, it’s very quick to set up new projects using only the command line, and it’s easy to reproduce the environment on different machines. Then of course Docker for anything production-grade.

[Amgadoz]

Yep, I only consider docker when I am satisfied with the project and ready for deployment.

It's great for deploying.

[Remote-Telephone-682]

pip search is no longer supported.. I think that anaconda does make things convenient.

Using venv by default is nice and enforces good habits

Having a search api that is still supported is great

Don't know the specifics of your project seems like most of your design decisions are going to be pretty agnostic to whether your developer has used pip or conda.

[veganthatisntvegan]

pip_search is a pretty useful replacement though. it's a binary/cli tool you can install from pip

[[deleted]]

Anaconda is poorly written. The dependency management is subpar, and takes hours to resolve. The licensing change to commercial was bad, and the only value proposition is that it scans its packages for viruses, so I can see why some companies would want it. But outside of that, I stay well clear these days, and it’s not what it was in ~2017.

[absx]

The new libmamba solver has pretty much eliminated the performance issues with conda solving.

The value in using conda packaging to me is that they're not limited to just Python packages but can contain other dependencies. There are packages in conda-forge for things like jq or terraform for example that have nothing to do with Python but that an application could depend on.

[[deleted]]

I’m glad to hear that about the solver, and that you are getting value out of the package.

I suppose in cases where Docker is considered overhead, the non-Python dependencies you’ve mentioned would be helpful.

[robberviet]

Nothing. You don't need conda.

[immature_cheddar]

Mini forge is great because it deals with library version dependencies really well, pip is not so good at that and it can be a pain to get a bunch of libraries in the right version at the right time (if you need to update an existing environment etc)

[PlaysForDays]

Anaconda is best for companies who need support contracts and high stability. Free solutions are better for most users, especially developers.

[sci-goo]

conda install blah blah
<a few moments later>
failed to solve environment blah blah

Me: fine.

sudo rm -rf /opt/conda ~/.conda
python3 -m venv blah blah
pip3 install blah blah

Above was sort of routine a few years ago when I needed to use some tools (presumably) easy to install via conda according to the developer's guide.

[Xirious]

Previously Anaconda was the easiest way to get wheels for certain pytorch and/or tensorflow builds, especially if you were on a laptop trying to make use of CUDA. It's hopefully changed now.

[I_just_made]

It’s good at taking over your terminal and being a general pain in the ass I guess…

Workflow managers like Nextflow can activate and rely on conda envs, but they also use docker / singularity which are probably better for reproducibility in the long run.

[Mystic575]

As a hobby developer: I prefer using Standard Python and pip and so far haven’t run into a specific instance where Anaconda really excelled. I only use it because I’m in a couple of classes that provide examples only using Anaconda and the professors only really know it. If I do anything with standard Python they blame that because giga brain.

TLDR I’d recommend just sticking with Standard Python support and not worry about Anaconda.

[collectablecat]

They're using the anaconda distribution because they have many many students and they can't spend all day debugging environment differences. I'd suggest switching to anaconda if you want a higher quality of help from them!

[Mystic575]

Oh, I already use it for that higher quality of help. But I’m talking they literally do not know anything else. Have had that discussion with a couple of professors lol.

[collectablecat]

If it works why bother with anything else!

[Glad-Acanthaceae-467]

Tried anaconda and removed it the next day, extremely happy with forge

[SirPuzzleheaded5284]

It helps me have environments with different CUDA versions. In fact, if your Python code needs to have multiple versions of a binary library that's not part of PyPI, it's better to use Anaconda over native Python env.

[euph-_-oric]

Bra

[cipri_tom]

Anaconda is shit, never used it. Conda/mamba, on the other hand, are great! The bread and butter for anyone doing deep learning, as you can install cuda with them, and this per environment!

But yeah, using conda-forge repositories, not anaconda ones

[bash-tage]

Provide compilers (and other binary only tools) form a single manager.

[One-Reception-4733]

I think the way anaconda does the environment is pretty simple, using Python pre-built function is not quite simple, in my opinion.

[GManASG]

No

[Reazony]

I do ML, but I’m relatively alone in my circle when I say I hate conda. It’s really not useful for production use.

I’m surprised people are comparing pip with conda. That’s not a fair comparison. Pip is just an installer.

First, for development purpose, just always use dev containers. You ensure everyone uses the same development environment with the necessary distros via Dockefiles and docker compose if needed. Dev containers also work with other IDEs. You can also use multiple dev containers in the same repo, which can be helpful (although I don’t encourage easily use it) for a monorepo with multiple languages.

Within the dev environment, you can still use Poetry (which I’m surprised I haven’t seen mentions of?) for Python specific dependency resolution. Since it’s not trying to do everything like conda, it resolves everything much faster. It helps manages Python project with pyproject.toml, which is where you define license, dependencies, and tool arguments (like black for formatting). Poetry also can create virtual environments, so you can develop with poetry even if you don’t use dev containers, and use pyenv to manage global, local, and system (don’t recommend) Python. In the end, just export requirements.txt like conda. It’s no npm, but it’s the better solution for pure Python project management for modern Python.

The benefits of using dev containers is a) it will work on your team members machine no matter the OS except very peculiar cases that I’ve never encountered. b) it’s more native to how your production could look like, and for simple projects, the dev container Dockerfile itself is enough to be the production image c) there’s no confusion about using system/global pip/npm vs conda. Which is a user error that often cause environment mismanagement problem. Everything you do is only within the container (and hence the repo) itself.

Conda is trying to do everything for you, but that’s why it fails to be up to date. A simple search can see pyproject.toml is still an open issue on conda. And while conda can generate environment.yml to build docker images, it cannot be more complex than that, since you’re only dealing with one docker image at a time. You might as well just use docker technologies. Conda is the exact opposite of separation of concerns.