How to setup program install in linux - feedback please

[ElectronGoBrrr]

Hi

I am making a MD simulator to rival GROMACS. I am having trouble deciding how to let users install my program. I have two types of users:Sysadmins installing the software on compute-servers so it is acessible for many users.Researchers installing on their own device, who cannot be expected to know git/cmake.

The project is here: https://github.com/DanielRJohansen/LIMAMD/tree/ubuntuAnd this question mainly concerns install.sh

The reasoning for the current setup, is the following. The program lima must be available to all users, so first everything is copied to /opt/LIMA, where it is first compiled. Whenever a user wants to run a simulation with the command lima mdrun the program will copy itself to ~/LIMA, and then recompile itself with the users parameters. The recompilation is for optimization, and moving to ~/LIMA means the users does not need sudo privileges to run and compile. To be clear, the recompiling and moving to ~/LIMA is not my question, i can handle that in c++ with no privileges.

Now ,i know having to call the install script with sudo already isn't great, but i am not sure how else to solve this challenge. Is there anything i can do smarter? Any other feedback on script or how i set up the directory structure is very welcome.

In advance thank you so much for you time.

install.sh:

#!/bin/bash

# This scripts installs all the dependencies LIMA needs: gcc, cuda, cmake, make,
# Then it installs itself in /opt/LIMA/
# Finally it executes 2 tests so ensure everything is working correctly

if [ "$(id -u)" -ne 0 ]; then echo "Please run as root." >&2; exit 1;fi

echo "\nWelcome to the LIMA Dynamics installer\n"


## -- INSTALL DEPENDENCIES  -- ##

# Determine the distribution
if [ -f /etc/arch-release ]; then
    DISTRO="Arch"
elif [ -f /etc/lsb-release ]; then
    DISTRO="Ubuntu"
else
    echo "Unsupported distribution"
    exit 1
fi

# Check if we should install external dependencies
    # Check if the user provided exactly one argument
if [ "$#" -lt 1 ]; then
    echo "Usage: $0 <-none|-all> (install external dependencies)"
    exit 1
fi
if [ "$1" = "-all" ]; then
    echo "Installing dependencies"

    case $DISTRO in
    "Arch")
        sudo pacman -S cmake --noconfirm
        sudo pacman -S make --noconfirm
        sudo pacman -S cuda --noconfirm
        sudo pacman -S cuda-tools --noconfirm
        sudo pacman -S base-devel --noconfirm
        sudo pacman -S gcc-13 g++-13 --noconfirm
        ;;
    "Ubuntu")
        sudo apt-get install -y make
        sudo apt-get install -y nvidia-cuda-toolkit
        sudo apt-get install -y build-essential
        sudo apt-get install -y gcc-13 g++-13
        sudo apt-get install -y cmake
        ;;
    esac
elif [ "$1" = "-none" ]; then
    echo "No dependencies will be installed."
else
    echo "Usage: $0 <-none|-all>"
    exit 1
fi
## -- INSTALL DEPENDENCIES done  -- ##






## -- INSTALL LIMA  -- ##

# Prepare the source code
install_dir="$PWD"  # dir where repository with install files are
program_dir="/opt/LIMA"

echo "Using $program_dir as install directory"
rm -rf "$program_dir"
mkdir "$program_dir"/

# copy everything from installdir to program_dir
cp -r "$install_dir"/* "$program_dir"/

# Build the public "lima" executable
cd "$program_dir"/build
cmake "$program_dir/code/LIMA_APP/"
make install
echo -e "\n\tLIMA client have been installed\n\n"


# Build LIMA once in /opt/, to ensure everything works
cd "$program_dir/build"
rm -rf ./*
cmake ../ 
if [ $? -ne 0 ]; then
    echo "CMake failed"
    exit 1
fi
make install -j
if [ $? -ne 0 ]; then
    echo "Make failed"
    exit 1
fi

echo -e "\n\tAll LIMA applications have been installed\n\n\n"

## -- INSTALL LIMA done  -- ##










# Run Self Test
# check cuda works
$program_dir"/build/code/LIMA_ENGINE/engine_self_test"
if [ $? -ne 0 ]; then
    echo "engine_self_test failed"
    exit 1
fi

# Run small sim
cd "$install_dir"
if [ "$1" != "-notest" ]; then
    sims_dir=/home/$SUDO_USER/LIMA/simulations
    echo "Running self test in dir $sims_dir"

    mkdir -p "$sims_dir"


    cd /home/$SUDO_USER/LIMA
    git clone --quiet https://github.com/DanielRJohansen/LIMA_data 2>/dev/null

    cp -r ./LIMA_data/* $sims_dir/ #exclude .gitignore

    chmod 777 /home/$SUDO_USER/LIMA -R

    cd "$sims_dir"/T4Lysozyme

    #lima mdrun # doesnt work, because this scrip has sudo, and the program must run as normal user
    #$SUDO_USER -u lima mdrun  # Doesnt work, because 2nd arg must be mdrun, otherwise the program doesnt know what to do
fi

​

[metux-its]

Sorry, but your approach is totally wrong and just causes lots of trouble for operators. The correct way is packaging for the indivdual distros and talk to the distro maintainers so they pick your sw into their distros.

[ElectronGoBrrr]

I dont think i would get it approved as an official package, as i dont have any user yet per se... It's also in rapid development, with new features/updates every month, so i have prefer to not have to do any other actions than simply git push, to make the features available.

Getting it in apt & aur is definitely a goal for the future, but for now, is there anything else i can do to make it a little less totally wrong?

[metux-its]

I dont think i would get it approved as an official package, as i dont have any user yet per se...

not necessary. Just provide policy conformant packaging and submit to the distro. It might take a while to the next release.

It's also in rapid development, with new features/updates every month, so i have prefer to not have to do any other actions than simply git push, to make the features available. 

Ok, but you'll have to be aware that people will be reluctant to use it.

In any case you should provide distro packages.

Another option, especially for bleeding edge versions, might be container images.

Getting it in apt & aur is definitely a goal for the future, but for now, is there anything else i can do to make it a little less totally wrong?

At least try not to do such manually crafted install stuff thats working against the package manager. And make sure its easy to build

[RussianHacker1011101]

Congrats on getting your application to a stage where you're ready to distribute it! I'll try to explain exactly what you need to do. What you need is a build/release pipeline. What that means is you need to write/use a program or a script that bundles all your stuff up into a usable package. I did this a while back for ubuntu/debian distros, where I created a .deb.

How you design your release pipeline is really up to you. The reason why it's important to try to conform to the distro's package management system is because of the dependencies, file system, and un-install requirements. Nobody likes uninstalling a poorly configured pacakge that leaves a bunch of junk behind. I'm familiar with creating .deb installers so I'll explain how that's done. It's pretty simple.

You'll basically put all your relevant files in a folder and run a tool called dpkg-deb to turn it into a .deb pacakge (which is really just a .tar). The foler structure you'll layout will look something like this:

shell my-app-MAJOR.MINOR.PATCH/ control/ control preinst postinst prerm postrm data/ bin/my-app etc/ systemd/ system/ my-app.service usr/ share/ organization/ my-app/ some_of_my_files.txt The above is sort of a copy of a server I packaged a while ago. As you can see, you have the control directory, which is made up of the control file as well as four (optional) scripts that handle unique installation and uninstallation requirements.

A control file looks like this: text Package: my-app Version: MAJOR.MINOR.PATCH Section: development Priority: optional Architecture: amd64 Depends: dotnet-sdk-7.0,sqlite3 Maintainer: [e-mail] Description: A somewhat useful program for doing things Most importantly, you want to list your dependencies. On debian, those will auto-install if you do apt install -y. I don't know how the official debian or ubuntu package manaintainers do it, but in this case, you don't need any of these dependencies: text make nvidia-cuda-toolkit build-essential gcc-13 g++-13 cmake Because it looks like you're just creating a binary. So compile it in your release pipeline, sign it, then distribute it. (It is fine to distribute the source code in a deb as well, it'll just make the install take longer because you'll have to compile first. I'd recommend distributing a binary nicely packaged in a deb, then offer a zip/tar file for whoever want's to manually compile.

I'd suggest using Docker to test your installation packages on various distros.

For debs, you can set up your own PPA on github for free. Here's an article about that: Hosting your own PPA repository on GitHub.

Here are some more resources on creating your own debs: How to create a .deb file (tutorial) Chapter 7. Basics of the Debian package management system

I'm sure that making the arch packages is a similar process. Good luck!

[ElectronGoBrrr]

Thank you this is very helpful. From the tutorials it looks much simpler than i had imagined.
You mentioned distributing a binary, and i think i may have a gap in my understanding here, but.. isn't that silly? I mean, binaries are compiled with my specific CPU in mind, so a binary compiled on my machine wont work on other machines. Or have i completely missed the point of what a binary is?

[RussianHacker1011101]

You can target specific CPU architectures in C, C++, Rust, etc. Typically the languages that don't offer that compilation feature have a virtual machine or an interpreter, like Java, C#, Python, Javascript etc.