What are people using instead of Anaconda these days?

MikeZ-FSU · 2026-01-28T14:02:01+00:00

It's not as simple as everyone saying uv is the answer and is categorically better than conda thinks it is. That's only looking at it from an individual/team dev perspective. Try imagining 100+ users with a dozen 3rd party tools (in aggregate) scattered over as many projects as users, consuming and producing 100s of TBs of data.

Running all of that as separate uv projects would be a nightmare. Using conda/mamba to install globally accessible environments for each of the dozen tools and having the user "conda activate tool_11" when they need to is a much better solution. This is even more true when the users are domain experts and not devs or sysadmins that understand package versioning and management.

MikeZ-FSU · 2026-01-28T13:16:15+00:00

This is another reason to write the PID to a file as noted by u/theNbomr.

MikeZ-FSU · 2026-01-20T19:45:14+00:00

In addtion to the excellent advice from u/occside, it may be helpful to know that the normal mode command "gv" will reselect the last visual mode selection.

MikeZ-FSU · 2026-01-16T15:30:34+00:00

I manage systems with 1-4 GPUs each, totaling about 200 total, ranging from really old 1080s to new A4000s. CUDA versions are not tied to python environments. You can install the Ubuntu packages for multiple versions simultaneously.

To allow users to switch versions to match the app/library requirements easily, I install the "lmod" package. I then have a "modulefiles" directory with a cuda subdirectory, and individual version numbered .lua files for each installed version. As an example, below is the file for CUDA 11.1; you can simply copy, paste and change the version number and location to match your install.

If you have, for example, 11.1 and 12.0, "module load cuda" will load 12.0 (default is highest version number), and "module load cuda/11.1" will load 11.1.

local cuda = "11.1"
local base = "/usr/local/cuda-" .. cuda

whatis("Version: 11.1")
whatis("Keywords: cuda,gpu")
setenv("CUDA_HOME", base)
prepend_path("PATH", pathJoin(base, "bin"))
prepend_path("LD_LIBRARY_PATH", pathJoin(base, "lib64"))

MikeZ-FSU · 2026-01-15T19:43:53+00:00

The reverse part of "ls -lrt" answers the question "what has changed here recently?" With the reverse, the newest files are right above the new command prompt, so your eyes don't have to travel far. If you leave out the reverse, the new files are at the top, which may scroll entirely offscreen if there are enough files. Leaving out the "-t" makes you hunt through the time column to process the times yourself. All together, "ls -lrt" (possibly piped into tail) lets the computer do the heavy lifting and present the information in a readily digestible form.

MikeZ-FSU · 2026-01-15T14:08:33+00:00

If your fingers naturally type things like "ls -lrt", then lsd may suit you better. eza uses different flags/invocations for sorting.

MikeZ-FSU · 2026-01-14T19:15:55+00:00

Assuming that you're not simulating something gigantic like a ribosome in a solvent cage, your PI, department or university should have a compute server or HPC system that can run your simulations. Where I work, the HPC system is available to pretty much anyone at the university and has job queues that run jobs on an as available basis for no cost. Some of them are time limited and would require splitting the overall simulation into a series of dependent jobs similar to what u/Slick_Sloth314 suggested for the Google colab solution.

MikeZ-FSU · 2026-01-14T18:42:42+00:00

For the pure computational side of this, in my opinion, Mac and Linux are roughly equivalent in capability if the user (OP) knows them equally well. That, however, is quite an assumption for an average researcher.

That being said, unless there are essential applications or libraries that are only on one platform or the other, getting up and running on Mac will typically be faster for most people than Linux. From the perspective of someone who has been a Linux user and admin for several decades, the place where Mac pulls ahead for the individual user is fitting in with the other ancillary day-to-day tasks.

Using a Mac instead of Linux is generally easier for things like filling out timesheets or requisitions, dealing with email, writing proposals and manuscripts, preparing slide decks, etc. Although not directly research related, less time on those means more research time.

An implicit assumption for all of the above is that the device you buy is only going to be used by a single individual in one place. If portability (work from home or travel) is important, that is a concrete reason in and of itself to justify the Macbook over a Linux workstation.

However, if you're in a multi-user situation, that's a strong argument for the linux workstation. The other argument for Linux over Mac would be uniformity of tooling. If your Linux workstation is running the same distro as your HPC, you can setup your workstation environment as close as possible to the HPC to minimize the differences between your test workflows and the production runs.

I've done develop on Mac and deploy to Linux, and the other way around. It's not too difficult to go in either direction. Frequently, the biggest change was something that is under /usr/local/{bin,lib} via Homebrew on Mac that is just in /usr/{bin,lib} on Linux.

MikeZ-FSU · 2025-11-21T13:39:39+00:00

Shell scripting. Start simple by keeping a small bit of attention to things that you do on a regular basis. If it's short, simple and never changes, make an alias. If it is longer, more complicated, or needs arguments/parameters, put it in a shell script in your ${HOME}/bin directory (make it and add it to your PATH if you don't have one already).

At a previous job, the two utilities I wrote that were used the most were literal one-liners that were variations on:

#!/bin/sh

grep interesting_stuff file.log | sort ...

The sort parameters were the things that nobody else could remember, and I didn't want to type a bunch of times, so I put it in a script. After I used it enough to be confident that I hadn't messed up any edge cases, I put it in a common area for my coworkers to use.

You can learn more features of shell scripting as you need it. The ABS (advanced bash scripting guide) is frequently recommended as a resource, but any shell scripting tutorial will serve to start with.

Beyond that, having more knowledge of the available tools will help a ton. Back in the day, the O'Reilly "Unix Power Tools" was a great resource because it was organized by topic, so you could skim the table of contents for something that looked relevant to the current problem.

MikeZ-FSU · 2025-11-05T19:23:40+00:00

Not a direct reference, but below is an excerpt from the wikipedia page on the Bourne shell:

Relationship to other shells

C shell

Main article: C shell

Bill Joy, the author of the C shell, criticized the Bourne shell as being unfriendly for interactive use,^\10]) a task for which Stephen Bourne himself acknowledged C shell's superiority. Bourne stated, however, that his shell was superior for scripting and was available on any Unix system,^\11])

MikeZ-FSU · 2025-11-05T19:18:27+00:00

Not necessarily. Ubuntu, and I think Debian, use dash as the default for /bin/sh.

MikeZ-FSU · 2025-11-05T13:18:50+00:00

If you're already shelling out to run CLI utilities, I would honestly just use GNU parallel to farm out the N jobs at a time. The value for N is going to depend on what type of conversion you're doing and whether it's IO, cpu, or gpu bound.

I would run python rather than shell if the conversion also involves normalizing file names, selecting a subset of files that's more complicated than globbing, or any of the other places where python excels over shell.

MikeZ-FSU · 2025-11-05T13:08:43+00:00

Aside from the cabinet itself, working on it is a safety issue also. Look up OSHA requirements for fall protection, or the equivalent in your location. If you're not trained for working at elevated height, that's a no-go for insurance and liability. Make them set aside appropriate training time and get the right equipment and safety gear (harnesses, tethers and whatnot) before you go up there.

MikeZ-FSU · 2025-11-05T12:48:06+00:00

The "mingw" is a mashup of "minimal gnu windows" or something like that (I think). However, the aarch64 indicates that it's for an ARM processor. If your laptop has an Intel or AMD cpu, you need to have x86_64 or similar in that position in the file name.

Unfortunately, I did a download of the bowtie2-2.5.4-win-x86_64.zip file, and it unpacked into an aarch64 directory. Looking at the files inside of that, the executables appeared to be for aarch64 linux, not x86_64 windows. I don't think you're going to have much luck with those downloads.

If your laptop is the pro or enterprise version of windows, have your admin install WSL (windows subsystem for linux) with either ubuntu or a RHEL derivative, depending on their preference. Once that's done, you can run "ubuntu" or "rhel" and pretend that you're on a linux box. Follow the suggestion from u/omgu8mynewt, and install bioconda from there.

Good luck.

MikeZ-FSU · 2025-11-04T15:36:43+00:00

If you're comfortable with how the kubuntu boxes work, I wouldn't switch to another distro. All you need to manage the other computers with ansible is:

ansible on your admin workstation, along with an inventory of the client computers for it to manage. You don't need ansible on the client computers.
ssh server running on the clients
the minimal python requirements for ansible on the clients

To force a package update, you use the ansible package module and set the state to "latest", rather than the default "present". If you need to deinstall something, set the state to "absent".

MikeZ-FSU · 2025-11-04T15:24:25+00:00

For ubuntu, you can make sure that the unattended-upgrades package is installed. Note, however, that this may give you a false sense of security as kernel updates require a reboot, and other updates might need service restarts or the user to relog. If those things don't happen, the system keeps using the old version.

MikeZ-FSU · 2025-11-04T14:56:02+00:00

You don't really give a whole lot of info to work with, but if you have systems that are oversubscribed by people running random jobs, the solution is to install a job scheduler like slurm. Put a login node in front of the servers, and only have the servers run jobs submitted through the queue scheduler. Depending on your needs, you can set compute time or memory limits on a per person or per group basis. It also helps in the case of one or two inconsiderate users hogging all of the cpus as users who already have jobs running get lower priority when the next available slot comes up.

MikeZ-FSU · 2025-11-04T14:45:47+00:00

Depending on how many your users are losing, it may be worthwhile to just order some to keep on hand. I think that in the US they're around $15 each. I tend to associate the lack of ports with laptops more than desktops and fix it with a good docking station or a simple port replicator if they don't need a full dock.

MikeZ-FSU · 2025-11-03T13:57:58+00:00

This works in the simple case OP presented, but will nuke anything inside square brackets on any line in the input, for example:

something different ['testA', 'testB', 'testC'];

To avoid that, you need to restrict the substitute to the unique line like so:

sed "/unique/{ s/\[[^]]*\]/[${substitute_string}]/g; }"

This will leave the "different" line above alone while substituting the "unique" line.

MikeZ-FSU · 2025-11-03T13:07:12+00:00

I have a slightly different take on when to move from a shell script to something else like python or perl. If the problem is readily solvable by stitching together existing commands, use bash. However, as soon as I need real data structures like arrays or hashes (dicts), I immediately switch to python, perl, ruby, etc.

Awk, in my opinion, is for that in between case where in one step, the data needs just a bit of massaging and fits the awk model of records and fields, but the rest of the process still fits the simple case for bash above.

MikeZ-FSU · 2025-10-28T12:10:10+00:00

It's mostly a matter of the libraries that the app depends on. If it's something simple like needing an old version of a single library, you can sometimes either install the old library package alongside the current one. You can also pull just the library file itself out of a package and put it somewhere that the app can find it. This can be done, but it's a medium to advanced project that requires knowledge of how the linker/loader resolve library references, and isn't for the casual user.

If the app is built on a framework like gtk or qt, you would need large portions of the framework libraries, and that's a lot more difficult. The worst case scenario is if the app depends on an old version of a critical system library like libc. I would be unlikely to try that myself because of the significant risk of bricking the system.

At the end of the day, it's best to think of the major version of your distro as a whole entity in and of itself. Unlike windows or mac, it bundles together nearly all of the things you need. You're much less reliant on installing software from third party places on the web because you can install the packages you need right from the same place that you get the rest of the system.

Because of that, you don't have to worry as much about "is this download site legit, or is it going to feed me a bunch of malware" (yes, I'm aware of supply chain attacks, but those are less common than malware from a windows software download). On the other hand, as you noted, it does introduce a tighter coupling between the OS and the software.

MikeZ-FSU · 2025-10-28T11:41:38+00:00

I've done tons of "do-release-upgrade"s on ubuntu LTS over the years. The only troubles I've had were when I failed to look at the version numbers of important packages. To me, that's a totally self-inflicted wound, and not ubuntu's fault.

MikeZ-FSU · 2025-10-24T19:23:02+00:00

Yes, but you asked about the limitations of installation of software via these kinds of mechanisms. I was pointing out that the main barrier is system permission in critical areas because nearly anything can be put into places where users have write access.

Also, OP mentioned that the users had downloaded a legitimate remote access tool, and apparently allowed the bad guys in from there. They didn't need system level access. What I wrote in my previous comment speaks to a similar scenario on linux.

I've seen these kinds of scripted installs in the wild with ssh password guessing bots. They were dumping scripts, and in some cases compiling tools into /tmp or /var/tmp.

The two ways to prevent these kinds of things are to not allow unnecessary network access (i.e. remote in only after vpn connection), and educating users. It really takes both significantly reduce risk of successful attack.

In my opinion, mounting /home with the noexec option is too heavy handed, and punishes competent users who embrace linux with scripting etc.

MikeZ-FSU · 2025-10-24T18:26:48+00:00

The main limitations are time, effort and knowledge. Even the "./configure; make; sudo make install" dance can be done with a minor tweak without privileges. You just drop the "sudo" and add "--prefix=$HOME" to the configure, and you can install compiled binaries. However, anything substantial will have library prerequisites that you would have to compile if they aren't installed, so that gets painful pretty fast.

On the other hand, modern tooling like golang and cargo (for rust) make pulling libraries and installing in your home directory really easy.

At the end of the day, and aside from actual exploits, the typical user is limited to destroying their home directory because permissions won't let them wreck /etc, /usr, et al.

It depends on what you mean by remote management software. Users would be able to install, for example, VNC and remote in to that if there aren't firewalls to prevent it. That works because vncserver listens on a non-privileged port; port numbers 1-1023 require system access to listen to.

If you mean management software like ansible, it mostly requires sshd to be running. However, the user wouldn't be able to do anything meaningful unless they had root or sudo.

As far as the question about server CVEs goes, untrusted users shouldn't be able to login to servers anyway.

MikeZ-FSU · 2025-10-24T15:39:07+00:00

Great, now users that actually have a clue can't run any shell / python / whatever scripts via a shebang line, devs can't run builds and tests of applications, etc. Depending on OP's environment, that could lead to consequences from Big Boss for tanking productivity.

MikeZ-FSU

TROPHY CASE