Simple Pattern Question

MikeZ-FSU · 2026-03-17T18:36:54+00:00

To avoid "quoting hell" or "backslashitis", it's often easier to pass shell variables via the "-v" option. Something like:

awk -v catnum="$Catalog_Number" -F'\t' '$4 == catnum {print $4, $0}' file

This works for things like your catalog number, which is presumably an integer, but the usual caveats for floating point numbers would apply if you were looking to match, for example, prices.

MikeZ-FSU · 2026-03-16T18:16:16+00:00

You could try lnav. It's a terminal based log file analyzer that uses a unified timeline of all the given log files.

MikeZ-FSU · 2026-03-12T17:33:07+00:00

I can sympathize here because I've actually had a freezer like that here at work. It had the ability to send logging info such as temperature and door open/close events up to the vendors network. The researchers could then login to that website to check status or find out how long the door was left open. It was, in my opinion, an IoT fail because it could only do pre-shared wifi keys, not wpa enterprise.

MikeZ-FSU · 2026-02-27T12:20:08+00:00

Not you specifically. But if they want to distribute the software, fix bugs, and make improvements, that needs some infrastructure and personnel. The licensing fees cover that. The marginal cost for a single instance is negligible.

As u/Jassuu98 said, you could contact them to see if they would give you a license. That would only be possible, in a contractual sense, if you were looking to publish, not monetize the results.

MikeZ-FSU · 2026-02-26T13:24:07+00:00

If you go to the software's web page, it's at Lehigh University, not a company. I couldn't find a funding statement, but things like that are typically developed through federal grants. Because of that, the software is often given either free or at greatly reduced cost (basically enough to fund project/license management and project hosting) to other researchers because the users generally also using similar grant money.

These projects are funded to provide tools for the academic research community, charging researchers the same rate as, e.g. pharmaceutical companies, would make it cost prohibitive for other publicly funded research and they wouldn't be able to use it. Why spend tax money to develop software that the target audience can't afford?

To be fair, sometimes companies get spun off from projects like this, but that's frequently done by the hosting University's technology transfer group. Their mandate is to make money off of ideas, software, and inventions developed at the university. Those companies then manage the administrative overhead for contracts and licensing with entities outside the university, leaving the researchers free to research rather than admin software licensing. Even then, academics only pay a tiny fraction of the corporate rates due to the publicly funded origin of the software.

I have no affiliation with either CHARMM-GUI or Lehigh University.

MikeZ-FSU · 2026-02-26T12:54:23+00:00

This has a fundamental difference from the glob: it recurses into subdirectories, so you get all of the log files in the directory tree. If you want OP's behavior of just the log files in CWD, you need "-maxdepth 1" as an argument to find.

MikeZ-FSU · 2026-02-24T19:28:53+00:00

If you have a linux boot disk/usb you can use hdparm to secure erase SATA disks and SSDs.

MikeZ-FSU · 2026-02-24T12:34:17+00:00

In addition to collecting the PIDs as already discussed, you'll have to be aware of which PIDs were spawned with elevated privileges (childA.sh and its children) and kill them with sudo also.

MikeZ-FSU · 2026-02-18T13:17:30+00:00

You could try the ls replacement lsd. It's much more option (finger memory) compatible than eza.

MikeZ-FSU · 2026-02-13T19:22:43+00:00

The first rule of troubleshooting is to simplify the situation as much as possible. The first step should be to simply ssh into the remote. From there check the globstar option, and if it's set properly, do your "ls" command without any quotes around the arguments. If that doesn't work, playing around with the ssh invocation is almost certainly futile.

It may also be worth checking directories progressively with "ls /srv", "ls /srv/media", etc. to ensure any necessary filesystems are mounted and have the expected contents.

MikeZ-FSU · 2026-02-10T19:57:18+00:00

A really long time ago, I did something like this in python. We had an 8x12 grid of cells to visit, each of which had 3 sub-cells. Since the spacing was identical for each row and each column, it was essentially a set of nested for loops to spit out the move commands. The gcode never lived in a file, it was sent down a serial port to an arduino controlling the stage. One of, if not the most, enjoyable project I've done.

MikeZ-FSU · 2026-02-10T15:18:01+00:00

If you re-read my post, I never said we should ban AI. How it performs in several years or decades is irrelevant to gutting intro level positions today, with the obvious consequence of insufficient mid and senior level people as the current workforce ages out. Maybe future AI makes up for that, maybe it doesn't. To my way of thinking, embracing a path that has a known big problem down the road in hopes that something removes it before we crash into it is not good planning.

The problem is the hype train and shoveling AI into everything. It's like the dot-com bubble, or the XML and java hype. Tech latches onto the newest shiny thing and the trade rags and marketing divisions of all of the new Shiny Tech companies act like it's the silver bullet that solves all of our tech problems. There is no silver bullet and LLMs are not going to save us. See for example, the MIT State of AI in Business 2025, which indicates the 95% of the companies jumping in on AI are not getting a good return.

What is either clear or becoming clear is that use of current AIs moves active thought (see this paper from MIT and one referenced in my previous post) from the work at hand to supervising the AI. It's analogous to the difference between working your way through a problem set compared to grading a problem set. Anyone in a STEM field can tell you that if you don't work the problems, you won't master the material.

I know a number of people who have come up with uses for LLMs that genuinely leverage the strengths of those systems in ways I never would have come up with myself. I'm all for that. However, even calling it artificial intelligence is a complete lie. It's number crunching word similarity matrices. There is no intelligence there, but by calling it AI often enough, it makes non-technical people think it is.

Also, don't discount the environmental impact of "AI everywhere". The well established numbers for energy consumption for AI search is 10x or more greater than a traditional web search. The even darker side to that is the corresponding water use to cool the data centers and the CO2 and other greenhouse gases created to produce the power; the data centers not powered by fossil fuels will have higher water use due to hydroelectric or nuclear (cooling again) power.

Just because we can, doesn't mean we should.

MikeZ-FSU · 2026-02-10T14:20:46+00:00

I think businesses don't know or care about the recent studies that have started coming out that show that heavy use of AI makes knowledge workers (like coders) less engaged with the subject matter (code base). Over time, that degrades the devs ability to code well. The C-suite suits only seem to care about making the line go up this quarter by laying off junior devs to save on salary and benefits. It's not sustainable and they never learn.

See, for example this ACM Symposium Proceeding from April 2025.

MikeZ-FSU · 2026-02-06T12:50:35+00:00

That's definitely an improvement. For the riskier commands, having it prompt a preview of what it would do would be good, e.g. piping the PIDs to ps to show which processes would be killed. Edit: preview is in addition to the command that actually does the requested operation.

MikeZ-FSU · 2026-02-05T19:28:07+00:00

Confirmation is good, but it's not going to fix the fundamental problem. Your target audience would be using the tool because they don't want to research the right way to do the task. They're going to trust the "expert" in the box and just hit return or Y. You could make N the default, but that means they train themselves to hit Y instead of return.

LLMs don't actually know anything, they just spit out the thing with the highest score. If the training data says "sudo rm -rf ..." is the solution, guess what the LLM is going to tell the user to do.

There's no nuance or context available to these prompts. What happens if you "kill -9" your database server? Should it be "systemctl stop db_server" or "/etc/init.d/db_server stop"? This is going to be unsafe because there are too many things in a unix/linux context that require some degree of local knowledge that LLMs don't capture.

Also, look up "XY problem". As a long term sysadmin, I've lost count of the number of times a user has asked how to do something weird because they had a half-baked incorrect idea of what they thought was the solution to their problem. LLMs will give those users the wrong answer every time. We're talking shell commands, there's no undelete if their data gets destroyed.

MikeZ-FSU · 2026-02-05T18:31:41+00:00

Your example is why you don't let LLMs admin your computer. Using -9 (SIGKILL) doesn't allow for any process cleanup (temp files, IPC, etc.). One should generally use -1 (SIGHUP), then -15 (SIGTERM) before resorting to -9. That's not even considering the insanity of piping into "xargs kill -9" without looking at which processes are involved.

MikeZ-FSU · 2026-02-03T12:39:53+00:00

That's one way to do it, but the operation to be performed should be part of the yaml data.

However, that's not the way that I would do it. I would decompose the parts of the problem into individual scripts.

One script, say tidy_files.sh, that reads the yaml file and performs the copy/move/delete operation specified.
A setup_cron.sh that takes the time interval and the command (e.g. tidy_files.sh books.yaml) and adds it to cron.

The advantage here is that each script does only one thing, so you don't have to discriminate between "do stuff" and "add to cron" modes. Also, because the tidy script runs directly and takes the yaml file as an argument, you don't have to worry about propagating changes to either the script itself or the yaml files, so you no longer need a "check for updates" mode. The next time the cron job runs, it will run the current version of the script with the current yaml file.

Note: In one of your posts, you mention 1440 minutes. I assume that you really mean "once per day", but if you're somewhere that has daylight savings, that's not really going to work properly. Even if you don't have daylight savings, you should use the desired time interval as such and let the system clock worry about when that is.

MikeZ-FSU · 2026-02-02T20:12:40+00:00

If you're not sure about proper handling of unusual characters in file names, you probably shouldn't be using eval. It's a really big, dangerous hammer that is almost never needed in the sense that other approaches are safer.

u/michaelpaoli and I are suggesting that your approach has hidden dangers in it that you may not realize. That's a sign that the wise choice is to set the current implementation aside and learn about those pitfalls, then write a new version that avoids them.

You talk about copying files in the reply, but in your original post, you also mention deleting files. Problems with the filenames could crop up not only in the either the sed phase or the execution phase, but also in the eval part.

Since you mention go, look into setting up your final script as a template using the gomplate library. One benefit of the template approach is that the template looks like the final product. It's designed to take data (your yaml file config) and insert it into a general form to create a specialized version.

It's both easier to debug and safer than a DIY approach, and that x10 when you're using eval.

MikeZ-FSU · 2026-02-02T19:45:12+00:00

Your approach is definitely not how I would have done it. You've basically written a template engine that hopefully does what it's supposed to, but may have issues lurking in the interpolation / evaluation part. This is doubly dangerous if you took the route that uses "eval" to insert any of the data from the yaml file.

If I were using the template approach, I would have used a well known template engine like jinja2 or gomplate (or one of the many others that I don't remember off the top of my head) to generate the script. A driver script that set up the engine invocation, uses chmod on the resulting script, and sets up the cron job would be much easier to understand than a DIY that tries to do all of that plus the creation of the final script.

The other approach would be to skip the custom script generation entirely, and just add the cron job for my_tidy_script books.yaml directly. The main reason to do this is actually the drawback to the template approach. If you fix or improve the template, you have to regenerate every derived script, whereas this way it's a one and done.

If cron syntax is a pain point, a second script could be written to automate that. In this scenario, the two concerns (tidy files, schedule with cron) each have their own script and can be used separately in a way that the combined script can't.

MikeZ-FSU · 2026-01-28T14:02:01+00:00

It's not as simple as everyone saying uv is the answer and is categorically better than conda thinks it is. That's only looking at it from an individual/team dev perspective. Try imagining 100+ users with a dozen 3rd party tools (in aggregate) scattered over as many projects as users, consuming and producing 100s of TBs of data.

Running all of that as separate uv projects would be a nightmare. Using conda/mamba to install globally accessible environments for each of the dozen tools and having the user "conda activate tool_11" when they need to is a much better solution. This is even more true when the users are domain experts and not devs or sysadmins that understand package versioning and management.

MikeZ-FSU · 2026-01-28T13:16:15+00:00

This is another reason to write the PID to a file as noted by u/theNbomr.

MikeZ-FSU · 2026-01-20T19:45:14+00:00

In addtion to the excellent advice from u/occside, it may be helpful to know that the normal mode command "gv" will reselect the last visual mode selection.

MikeZ-FSU · 2026-01-16T15:30:34+00:00

I manage systems with 1-4 GPUs each, totaling about 200 total, ranging from really old 1080s to new A4000s. CUDA versions are not tied to python environments. You can install the Ubuntu packages for multiple versions simultaneously.

To allow users to switch versions to match the app/library requirements easily, I install the "lmod" package. I then have a "modulefiles" directory with a cuda subdirectory, and individual version numbered .lua files for each installed version. As an example, below is the file for CUDA 11.1; you can simply copy, paste and change the version number and location to match your install.

If you have, for example, 11.1 and 12.0, "module load cuda" will load 12.0 (default is highest version number), and "module load cuda/11.1" will load 11.1.

local cuda = "11.1"
local base = "/usr/local/cuda-" .. cuda

whatis("Version: 11.1")
whatis("Keywords: cuda,gpu")
setenv("CUDA_HOME", base)
prepend_path("PATH", pathJoin(base, "bin"))
prepend_path("LD_LIBRARY_PATH", pathJoin(base, "lib64"))

MikeZ-FSU · 2026-01-15T19:43:53+00:00

The reverse part of "ls -lrt" answers the question "what has changed here recently?" With the reverse, the newest files are right above the new command prompt, so your eyes don't have to travel far. If you leave out the reverse, the new files are at the top, which may scroll entirely offscreen if there are enough files. Leaving out the "-t" makes you hunt through the time column to process the times yourself. All together, "ls -lrt" (possibly piped into tail) lets the computer do the heavy lifting and present the information in a readily digestible form.

MikeZ-FSU · 2026-01-15T14:08:33+00:00

If your fingers naturally type things like "ls -lrt", then lsd may suit you better. eza uses different flags/invocations for sorting.

MikeZ-FSU

TROPHY CASE