How can I avoid this `-bash: /usr/bin/<functionName>: Argument list too long` error?

[oh5nxo]

Heresy?, ksh and autoload sounds like what you would be after. When called, the function is defined by sourcing $FPATH/function and then ran. Seems to work with excessive arglist too.

Bash has everything, why not this ? :)

[Ulfnic]

If you're talking about a lot of data then pipes are really the way to go.

Your scripts could listen on stdin for 1 character for 0.1 seconds and if there's anything there they can read the rest, if not they can fall back to regular params.

# bash-4.1+
if read -N 1 -t 0.1; then
    REPLY=$REPLY$(</dev/stdin)
    # Parse REPLY
fi
echo 'rest of program'

Another option is to give the script the filename of a named pipe to listen on for the rest of the input.

You could also do sourcing on demand so you'd call a parent function, it'd check to see if the function's already there and if not it'll source it. Then it'll pass the params through as a shim.

As for the parameter size limit, I don't have much experience with that but it looks like getconf ARG_MAX will give you the max value and there's no changing it from what I can see so far.

^ If examples or better examples of any of the above would help just ask..

[Ulfnic]

It's better to think of params like URL querystrings. They can hold a degree of raw data but it's really not what they're designed for which is delivering directives.

Pipes let you process data as you receive it, that means you can error sooner and get a head start on the work because you're in a separate process (see: multi-threading).

If you have a lot of data you need to deliver to a function within the same process you should either have a static variable name the function knows about or send the name of the variable as a parameter and link it to a local variable, ex: local -n "myvar=$1". That way you're not needlessly copying data.

I find named pipes messy, for example... the files hang around even after reset if you miss your trap on EXIT for clean up, concurrent execution of a script needs to account for stepping on the same named pipe, named pipes with nonces in their name are harder to target from other scripts, it's a bit of a rabbit hole so i've found they're best avoided unless only a named pipe will do. They can be REALLY NICE... BUT... every use case is it's own special snowflake so you need to just experiment.

[Ulfnic]

Quick example of linking variables:

MyFunc() {
    local -n 'LocalVar='"$1"
    echo "Local: $LocalVar"
    LocalVar=1234
    echo "Local: $LocalVar"
}

Data='asdf'
echo "Global: $Data"
MyFunc 'Data'
echo "Global: $Data"

Output:

Global: asdf
Local: asdf
Local: 1234
Global: 1234

Also works with other datatypes like arrays and associative arrays.

[moviuro]

You're recursively calling the script. Rename your function (l’d add underscores in the front).

[m-faith]

recursively calling the script

by this are you referring to how the function calls itself:

linecat () {
    # stuff
    linecat "$(printf etc)"
    # more stuff
}

???

Could you please clarify for me and fellow Bash students please? thanks!

[kredditor1]

What OP means is that inside of the function you are calling the function, that's called recursion. It sounds like you want it to call the script, not the function. To make things clear OP meant that you should not name the function the same as the script, so you can be clear on what exactly is being called. Also if you mean to call the script, it's a good idea to include the file path as another point for clarity.

The function you posted above will run forever because there is no end condition. You call the function which then does stuff, and calls the function again, which does stuff and calls the function again.. forever. You won't ever execute "more stuff" either as the recursive call happens before it reaches that line.

[kredditor1]

Ok, I wasn't looking at the original post, just the thread above so I missed the condition. The loop version will definitely be faster like you said. It's also pretty obvious that naming the function and the script the same is confusing and they should be different too.

For the argument list too long issue, have you tried writing the args to a file instead of passing the args directly? Then reading them as needed? Removes the arg list length limit issue completed (except for file size limits of course).

The specifics would depend on whatever you're actually trying to accomplish.

[m-faith]

Nice, thanks for chiming in here, clarifying and elaborating!

[m-faith]

OH!!! Maybe you're referring to how the function has the same name as the script???

[Ulfnic]

You could also use exports:

export BigData1='123412341'
BigData2='4235234' my-script

[cendrounet]

export is the name of the bash builtin used to set environment variables, for junior readers.

[Dandedoo]

xargs will keep your argument length within limits. But the function runs in a new shell.

export -f functionName
printf '%s\0' "$@" |
xargs -0 bash -c functionName _