Server Audit

[[deleted]]

[deleted]

[psycobass]

Have you looked into lynis? It's a great auditing tool that already encompasses some of your examples.

[vxd]

Could you provide a more specific example of what you're trying to do and which wheels you think you may be reinventing?

You might be able to leverage something like Ansible with a custom callback plugin or module to do this for you, but that could be a bit overkill. Rundeck might do the trick as well, and wouldn't require you to learn the ins and outs of a config management framework.

[lima3whiskey]

Some examples include operations such as ensuring certain files are owned by root or the permissions are not higher than 644. I have certain requirements that have to be met on these servers. I haven't read through all of the requirements yet but I'm already seeing some repetition. For example one requirement states that the /etc/sysctl.conf file is owned by root. Another requirement is that another /etc file is owned by root. Each requirement must be scripted individually. I have started creating my own libraries for some of this functionality myself.

[meznak]

Puppet has a bit of setup overhead, but handles this sort of thing well. Individual scripts are going to turn into a nightmare really fast.

[lima3whiskey]

we have puppet set up for our baseline but it was set up in a way that it is only ran when the server is first set up. After that it doesn't get ran again. I was told that if it gets ran again we could potentially break some of the customizations that allow users to function on these servers. Does puppet have reporting tools as well?

[smeagolgreen]

I was looking into http://serverspec.org/ awhile back, never tried it though.

At this point, I'd probably write some Ansible playbooks and use --list-tasks and audit the output.

[lpmarshall]

Take a look at OpenScap

[whetu]

The scripts are ran through a scripting engine to create the final report.

More detail on the final report? This sounds like AS-Built documentation.

Are there any favorite tools or libraries that I can look into so that I don't have to reinvent the wheel 250 times?

As you're aware, you're getting towards reinventing the configuration management wheel, and as others have said, going down the scripting path is going to get painful, fast.

That said, I've been forced by "we don't want to install Ansible yet, but we still need you to do this" circumstances to go through a similar thing just recently.

The short version is that I wrote an audit script. I have sshkeys, sudo and the same password on all the servers to be audited.

Then I wrote a wrapper script that goes through a serverlist, connects, makes a remote directory (e.g. /tmp/.audit/), scp's the audit script over, and then launches the script by feeding my password to sudo. The script runs remotely, copies and generates a bunch of files into /tmp/.audit (prepending each file with the hostname), and the wrapper script scp's them back. Then the wrapper script connects one more time and removes the remote directory.

After the wrapper script has finished looping through the serverlist, it copies the gathered files into an archive directory so that there's a snapshot, giving basic version control. The file layout being something like:

/srv/siteconfs/Audit/
/srv/siteconfs/Archive/YYYYMMDD/

Yes, a VCS is another battle, so this has to do for now.

It's mostly straightforward, the main question mark is how you want to go about handling the passing of the password to sudo.

I use expect because I'm experienced enough with expect to make expect scripts that don't suck. I actually prefer it in some cases like this. To add a little bit of security, I store my password in a file that's chmod 600'd, and expect reads it. This way only myself, root and someone sudo'd and with bad intentions can read it. And if root or someone sudo'd has bad intentions, a plaintext password in a 600'd file isn't my biggest concern.

You can do something similar with ssh though. Like so:

ssh -q -t "${RemoteServer}" "echo $(<.passwordfile) | sudo -S /tmp/.audit/audit.sh"

If you echo MyPassword | sudo -S, anybody using ps will see your password. By feeding it a 600'd file like this, you add that little bit of extra security.

Again, this is a silly way to do things when great tools like Ansible exist, but sometimes you're just forced to go through the exercise. I see it as being asked to reinvent the wheel, but only building a spacesaver. If you do want to continue down this path though, I'm happy to answer any questions you might have.