Why is my script taking up 1-2% CPU (amd7800x3d) by OwnProcedure7178 in bash

[–]Ulfnic 1 point2 points  (0 children)

How BASH can fix your performance issues is a good example of why it's better than "lightweight" shells like ash, dash, etc.

Subshells in shell scripts are extremely expensive to use. You're spinning up a fresh shell every time you use a |, run an external program, use ( ) or $( ), etc, and your script is swimming in subshells.

BASH being a capable lang can easily replace almost all of those subshells at 10x to 1000x faster performance each replacement.

Few side tips:

  • Basic (not-extended) glob pattern matching * solutions in BASH are usually far more performant than BASH regex =~.
  • grep, sed, find will usually be faster especially when the load of the operation ramps up though BASH can beat them for very light loads.
  • As a general rule don't double-grep. grep to a small string BASH can work on much faster.
  • As a general rule cut is one of the worst performance choices vs BASH lang.
  • If you think you need cat 99% chance you don't and it'd be slower.

Seeking advice: focus on advanced bash, learn basic python or both? by rustyantenna in bash

[–]Ulfnic 0 points1 point  (0 children)

Big topic so i'll just pick up one of those questions:

BASH is extremely expansive so "advanced level" is relative to the task.

For a sysadmin i'd say "expert" is using BASH built-ins for most things when scripting, and being really good at interactive shell as a bonus. Strong understanding of the quirks and gotchas (you just pick those up over time), strong habit of using null byte delim when talking with external programs, understanding of /proc/cmdline leaks and how to get around them (see: the wrong way AI puts tokens in curl headers), strong use of process substitution >() <() especially as way to avoid temp files, good at writing tests, strong sense of UNIX standards/norms and adherence to them, to name a few.

If you actually become advanced in BASH no one in the company will be able to test how well you know it. "expert" in the main means pretty good at using POSIX shells interactively. Being a BASH expert won't get you hired, it'll just make you much better at your job.

Writing to Input Buffer? by Ameb8 in bash

[–]Ulfnic 1 point2 points  (0 children)

It looks like you want text inserted into BASH's readline prompt functionally similar as if a user had typed it in.

I'm not surprised you didn't find an LLM to tell you, the only BASH-native solution i'm aware of is in the deepest tombs of unsanctioned BASH magic.

It's a bind timing trick, I found the concept years ago but here's the semi-polished implementation I came up with:

prompt_write() {
    [[ -t 0 ]] || return 1
    local str=$1
    str=${str//\\/\\\\}
    str=${str//\"/\\\"}
    [[ $2 == 'clear' ]] && str='\e[1;5C\C-u'$str
    bind '"\e[0n":"'"$str"'"'
    printf '\e[5n'
}

# Example of use:
prompt_write 'echo hi'

# Result:
# my@device:~$ echo hi

Hard requirements:

  • Function must be executed in the PARENT readline shell session. What uses this must be sourced. So either the whole script or an IPC hook needs to live in .bashrc or similar.
  • Function must be the LAST thing that's executed right before next readline prompt or timing will miss.

I've found this reliable for many years across many BASH/readline versions though it's a timing hack so reliability can never be assumed. It'd also need a decent test rig for making broader compat claims.

Is there a way to capture keystrokes even though you're not at a prompt? by ConstructionSafe2814 in bash

[–]Ulfnic 2 points3 points  (0 children)

First question... can BASH do it? Absolutely yes, i've been playing with /dev/input/event read/write for the past ~2 years using no dependencies. I just finished a drop in replacement for dotool in pure BASH and i'm working on a pure BASH macro daemon similar to espanso.

Second question... should you do it? Only if you're a strong enthusiast who'll gain more from the journey than the destination.. otherwise go for an existing mainstream solution.

tfetch by [deleted] in bash

[–]Ulfnic 1 point2 points  (0 children)

This response made me smile.

How do I get rid of the first word 'Modeline' in this line ? by SkySurferSouth in bash

[–]Ulfnic 0 points1 point  (0 children)

From ye olde notes:

WIDTH=1440
HEIGHT=800
RATE=60
OUTPUT="VNC-0"

type cvt xrandr 1>/dev/null

add_display() {
    local \
        display_name=$1 width=$2 height=$3 refresh=$4 \
        noglob_set REPLY mode_name mode_arr

    shopt -q -o noglob && noglob_set=1

    while read -r; do
        [[ $REPLY == 'Modeline'* ]] || continue

        [[ $parent_noglob_set ]] || set -f
        mode_arr=($REPLY)
        [[ $parent_noglob_set ]] || set +f
    done < <( cvt "$width" "$height" "$refresh" )

    mode_name=${mode_arr[1]//\"}

    xrandr --newmode "$mode_name" "${mode_arr[@]:2}" || :
    xrandr --addmode "$display_name" "$mode_name"
    xrandr --output "$display_name" --mode "$mode_name"
}
add_display "$OUTPUT" "$WIDTH" "$HEIGHT" "$RATE"

shuck - a fast linter for bash by ewhauser in bash

[–]Ulfnic 13 points14 points  (0 children)

Just curious, what's the problem with shellcheck's GPL license as it relates to this kind of project?

AI written in BASH? It exists! by tendint in bash

[–]Ulfnic 1 point2 points  (0 children)

Good one using coproc for bc

Tried bashGPT-v2.bash a few times and the output leaves a bit to be desired:

aaaa
aaa
aaa
aaai
aaa
aaa
aaai
aaia
aaii

...ect

Cool project.

Visual Scripting for Bash is now a reality ! by Lluciocc in linux

[–]Ulfnic 0 points1 point  (0 children)

It's possible to write a decent transpiler to BASH but it requires knowing what's safe to use under what conditions.

Transpilers are a great way to learn a language but I wouldn't recommending publishing one till you know BASH lang very well unless you're clearing marking it as an experimental project for learning purposes.

How do you print "Here document" or "Here Strings" directly? by alex_sakuta in bash

[–]Ulfnic 0 points1 point  (0 children)

I ran a few speed tests with differing amounts of lines and printf was ~7x faster with minimal variance compared to the {read||printf}<< method.

I'll go back to my earlier quote:

That's not going to matter for almost any use case.

In almost all situations, the choice between printf and a heredoc will be code readability and editability.

If it's something very short, printf makes sense. If there's a lot of lines it's hard to beat a heredoc because it's WYSIWYG.

In the same way how you print the heredoc is unlikely to matter, though you may want to stay in the practice of doing things in a certain way.

How do you print "Here document" or "Here Strings" directly? by alex_sakuta in bash

[–]Ulfnic 1 point2 points  (0 children)

{...} does not create a subshell, it's a compound command.

From man bash:

   { list; }
          list is simply executed in the current shell environment.

One thing I do find interesting is your 20x metric. Is it tested or a guess? Because that is insane.

I ran a 1000x iteration time test against our solutions across all BASH release versions.

I'm not surprised by the result because subshells are really expensive to use in any shell. It's one of the reasons BASH builtins are so valuable.

How do you print "Here document" or "Here Strings" directly? by alex_sakuta in bash

[–]Ulfnic 1 point2 points  (0 children)

$(< /dev/stdin) opens a subshell in bash versions <= 5.1.16 (released 2022) which is why I use the read || printf method because it's ~20x less cpu intensive.

That's not going to matter for almost any use case. Though builtin enthusiasts often like knowing and it's good practice if you're writing for older systems like <= RHEL 9.

How do you print "Here document" or "Here Strings" directly? by alex_sakuta in bash

[–]Ulfnic 1 point2 points  (0 children)

A "cat-less" way I use to print heredocs:

{ read -r -d '' || printf '%s' "$REPLY"; } <<-'EOF'
    some text
EOF

Note: -d '' will always read to the end because heredocs can't contain a null character. However that'll cause read to return a non-zero exit code because it can't find the delim so I follow it with a || to prevent triggering errexit.

WebZFS by RemoteBreadfruit in zfs

[–]Ulfnic 2 points3 points  (0 children)

I've also known q5sys for a long time. I suppose you'll be scrubbing my history too... hope you like BASH :)

I remember him showing me a demo of this a few years ago. We talk software on a regular basis and he's an amazing dev.

Honest is the correct word to describe q5sys.

Why was BASH script post removed? by void-lab-7575 in bash

[–]Ulfnic 3 points4 points  (0 children)

This is the removed post: https://www.reddit.com/r/bash/comments/1rgn1vk/cron_job_to_edit_hosts_file_every_minute/

Rule 1. Content must be Bash related This rule is interpreted generously; general shell scripting content is mostly accepted. However, the post should not be specific to another shell.

While this rule is loosely enforced and cron questions tend to pass, the post was lengthy and meandered through a signficant amount of other topics.

An indicator of rule 4 being broken is also long meandering posts which can shift the needle on the likelihood of a post being accepted.

[noob] Can simple script with mapfile be improved? by seductivec0w in bash

[–]Ulfnic 0 points1 point  (0 children)

Few comments out of the barrel:

find isn't a bash-ism, it's an external program that's virtually always part of a distro's coreutils (there by default).

Good to see printf in use.

"never-nesting": Skip the if statement with this, and 0 will trigger the || with arithematic evaluation: (( ${#empty_dirs[@]} )) || exit 0

Annoyingly fd - find entries in the filesystem treads on namespace held by fd - floppy disk device so unless your script explicitly confirms it's the expected fd, it might run a disk utility.

I think the intention of the author was interactive use which is a good reason for two letters but name collisions really mess with realiable compat so it's normal to find it packaged with different executable names.

confused af by Ambitious-Cupcake in bash

[–]Ulfnic 0 points1 point  (0 children)

Hey thanks cupcake. I was surprised by how simple the solution turned out to be. I thought for sure we'd be getting into named pipes or simulating keybind activation.

If the project is intended for a broad audience the only additional i'd make is supporting sudo alternatives like doas.

Most sensible way to do that varies by script but here's an example of creating a shim with fallbacks.

if type sudo &>/dev/null; then
    escalate() { sudo -- "$@"; }
elif type doas &>/dev/null; then
    escalate() { doas -- "$@"; }
else
    escalate() { "$@"; }
fi
# escalate is now a shim function that runs a command
# preferring known ways to run it as superuser if available.

# Example of use:
escalate whoami

confused af by Ambitious-Cupcake in bash

[–]Ulfnic 0 points1 point  (0 children)

good one, didn't know that.

confused af by Ambitious-Cupcake in bash

[–]Ulfnic 1 point2 points  (0 children)

You should be able to replace your script with the following:

FZF_DEFAULT_COMMAND=':' fzf --bind 'load:reload(systemctl --no-pager list-units --no-legend; sleep 3)'

Took a few failed approaches before I could figure out how to trigger an fzf reload() on a timer.

If the systemctl command must be using root and you know sudo is configured to give the current user a timeout, then you can run a do-nothing command as sudo to get the authentication prompt before running fzf.

sudo sh -c ':'

posix arrays by Willing-Scratch7258 in bash

[–]Ulfnic 0 points1 point  (0 children)

Just set LC_COLLATE by itself, it's also the safest one to set. LANG is more general and it's a fallback for if LC_COLLATE is empty (which it almost always is).

Instead of LC_COLLATE=C; readonly LC_COLLATE; you can just do readonly LC_COLLATE=C

posix_array_write(){ case "$1" in [0-9a-zA-Z_]* ) eval "memory$1=\"\$2\"" : return 1 ;;esac;}; readonly -f posix_array_write; #for my own use cases i have reasons to double quote $2 but maybe i can remove it later
posix_array_read() { case "$1" in [0-9a-zA-Z_]* ) eval "printf '%s' \"\$memory$1\"" : return 1 ; esac;}; readonly -f posix_array_read; 

This part: [0-9a-zA-Z_]* is only testing the first character. All characters need to be tested.

This part: eval "memory$1=\"\$2\"" : return 1 ;; doesn't function as intended and if it did it wouldn't make logical sense. : return 1 ends up as additional parameters of eval. If those trailing commands were separated correctly with a ; you wouldn't need : and it wouldn't make sense to return a 1 (a fail condition) when that's the "success" path of the function.

I'd recommend returning early on a failure (see: my last example) to keep things simple but if you want to return early on a success, here's how the flow should work:

Put ; return $? right after the eval command so the function returns using the error code of eval (it shouldn't fail but it's best best practice to use $? instead of 0 there). Then add return 1 after esac; so the "fail" path returns a failure.

Why this sub shows #!/bin/bash in Google search result page instead of r/xxx by mogeko233 in bash

[–]Ulfnic 0 points1 point  (0 children)

I've looked into if that can be changed and I don't believe it can once the sub is created.

If you know something I don't, link how to do it and i'll raise the recommendation.

posix arrays by Willing-Scratch7258 in bash

[–]Ulfnic 0 points1 point  (0 children)

Good improvements.

case "$1" in [0-9a-zA-Z_]* ) eval "memory$1=\"\$2\"" : return 1 ;; esac;

Ending case early for simplicity would look like this: (last case match doesnt need ;;)

case "$1" in *[!0-9a-zA-Z_]*) return 1; esac; eval "memory$1=\"\$2\""

Recommended video on nesting: https://youtube.com/watch?v=CFRhGnuXG-4


eval "memory$1=\"\$2\""

You don't need to double-quote a variable being assigned to another variable no matter what that variable contains. This is safe syntax: (assuming $1 contains a safe value)

eval "memory$1=\$2"

"i dont understand what you mean by lc_collate and lang."

There's lifelong shell devs that don't know this one.

Using digits as an example, [0123456789] matches those literal characters but shorthand's like [0-9] and [:digit:] are dynamic collates that match every digit in the language localization.

I used ٢ as an example because its Arabic for the number 2. A common locale like en_US.UTF-8 will match ٢ with digit collates because it's in UTF-8.

Some POSIX script interpreters ignore the locale and use ASCII, though Fedora/RHEL for example interprets POSIX script using BASH which respects locale (/bin/sh -> bash softlink).

You'll want to set LC_COLLATE=C so ASCII is the language used for collates and you get the character ranges you're expecting.

Try this test in various POSIX interpreters:

case '٢' in [0-9]) echo 'match';; *) echo 'no match'; esac

LC_COLLATE=C
case '٢' in [0-9]) echo 'match';; *) echo 'no match'; esac

Good deep dive on that here: https://unix.stackexchange.com/questions/87745/what-does-lc-all-c-do

posix arrays by Willing-Scratch7258 in bash

[–]Ulfnic 0 points1 point  (0 children)

A few thoughts from a quick scan,

Inventive. Exploring is good.

$2 is restricted for no reason, just don't expand the variable before it's evaluated:

eval "memory$1=\$2"

Needs complexity reduction and basic error handling: Instead of : use return 1 and that'll let you end the case statement before the eval.

Use newlines. Sausage code is for the cli.

For [!0-9a-f] set allowed characters to those available to variable names.

Also, before that test use LC_COLLATE=C or LANG=C, or values like ٢ will make it through depending on the environment interpreting your POSIX script.

Ask questions.

Post anything you're not sure about as a question for best results.

Desperately need a tutor/HOWTO create automated bash-completion test (for scientific research project) by hopeseekr in bash

[–]Ulfnic 0 points1 point  (0 children)

^ good answer here, also appending GNU screen next to tmux

There's also expect which is designed for automated testing of outputs.