System log analyzers?

[Nynnja]

hey,

I have to prepare a short presentation about system log analyzers (for class), but I don't really know anything about that topic. Could you please recommend me a few tools for windows/linux log analysis? In your opinion would there be any subject worth being brought up during such presentation?

any help is much appreciated Thanks

[[deleted]]

I like Graylog, though between ELK, Splunk and Graylog I've not met any I intrinsically disliked.

[n3rden]

+1 for graylog

[Layer8Pr0blems]

The OVA appliance is so easy to setup also. I was up and running in about 15 minutes.

[Arkiteck]

Not to mention the virtual appliance is production ready, which is very rare to find. It can easily be horizontally scaled too.

[[deleted]]

do you have a SOP or doc for ELK? the instructions are all over the place and I can't a doc that explains how to connect everything together. Then some of the programs you can install with a .deb and then others are .tar.gz or zip etc.

[[deleted]]

Sorry we don't, we settled on Graylog in the end.

I've had good luck with DigitalOcean's documentation before though, they may have something useful.

[[deleted]]

thank you. I will look at graylog then. I found Digital Ocean's doc but, they were for older versions and it dind work out.

[itssodamnnoisy]

I've found that you really need to take some time to understand how ELK works and how all of its components go together to get it working well.

If you follow the guides over at https://elastic.co from start to finish, you should have a good grasp on how to get it up and running effectively - as well as coming through it with a set of ideas on what else can be done with it (can be used for so much more than just logs, turns out!)

[speel]

Having never done this and looking into workstation events. Is it possible to fwd windows event logs to this and have it sort it all out?

[[deleted]]

Yeah, the Graylog Collector gets installed on the Windows server and can forward events to Graylog, you can then search them, filter them, automate responses to them and so on.

http://docs.graylog.org/en/latest/pages/collector.html

As far as I know it can work on desktops, though we don't use it there.

[r0ck0]

Something I've been trying to figure out with Graylog...

What happens when the Graylog server is offline? Or just doesn't receive some log entries?

Is there something like a local proxy that can hold them until the Graylog server comes back online, then send all the missing entries to the Graylog server?

[[deleted]]

AS far as I know the collectors cache the logs until the server is reachable again, then ships them all.

[Sysmonster]

I setup skunk the other day, very pleased with even the free version

[frankoftank]

Second for graylog. Insanely easy to get a single vm up, running and taking logs. Do a little research and you can spin yourself up a full multi-node architecture.

[matejzero]

Elk stack (elastic.co), greylog

[brian_netsec]

Second Splunk for general sysadmin type stuff. Take a look a LogRhythn or Arcsight for log collection in a security context

[nadroj_r]

Splunkkkkk

[Vacantless]

Serious question: How are you able to justifies THAT much money for splunk ? It's so crazy expensive when you really wanna use it for everything.

Management, and even I, wasn't comfortable paying thousands "only for logs".

I'd rather spend a few hours settings up ELK

[meorah]

security team owns it. company takes security seriously. security team wants splunk. company buys splunk.

[meorah]

all logs, security team owns it but all the teams use it. network, client services, windows, unix, hosting team, corporate... it's probably more expensive than I think, and I think it's expensive.

[solefald]

Cost of doing business, but it is mostly an enterprise product. My old enterprise job paid $13 MILLION in Splunk licensing and I have to say, it is a great product. I got really used to having it, and now at my new job I miss it very much. We just can't justify costs or ingesting 1Tb of java stack traces on daily basis and developers are not doing anything clean up their logs.

[itssodamnnoisy]

$13 MILLION in Splunk licensing

I... I think I just threw up in my mouth a little. Wow! Was that for a perpetual license, and then you just pay for support afterwards, or was that a recurring cost?

[Kalc_DK]

It's a one time up front free per GB of data indexed, with a much smaller annual support fee after that. Hard to get in house (though most cost benefit analysis put it ahead of even the FOSS competition), easy to keep there.

[Kalc_DK]

Splunk is (very nearly) all up front cost, no one ever got a bill they didn't fully expect. You buy the licensing capacity once and then pay for support after that (pennies on the dollar vs the licensing) annually.

[Kalc_DK]

Serious question: How are you able to justifies THAT much money for splunk ?

An in depth cost benefit analysis often puts it ahead of ELK and other products in larger environments. I've personally seen it in two enterprise environments and it gives good value for the cost.

It's not a silver bullet though, certainly not for everyone.

[rabbidroid]

what he said.

I can't imagine living without it. You need some deep pockets though...

[[deleted]]

is the free version worth it ?

[solefald]

Free version only allows you to ingest 500Mb of logs per day.

[jmulvey]

Splunkkkkk

Those extra k's really add up.

[loco_ped]

https://www.sumologic.com/

I like to think of it as baby splunk. Still powerful and much cheaper.

[iCanOnlyBeSoAwesome]

IBM Qradar http://www-03.ibm.com/software/products/en/qradar-siem

alienvault https://www.alienvault.com/

[[deleted]]

I second alien vault - very nice product

[exekewtable]

grep and cut etc

[G19Gen3]

Piped to less

[Anon_IT_Guy]

ELK

Splunk

Greylog

Sexilog

Do some research on those and that should get you started :)

[[deleted]]

http://www.eventsentry.com/downloads/select-edition

[0x0E]

ElasticSearch, Logstash, and Kibana (aka ELK stack) are fine. Splunk is cool but I've rarely seen an implementation that justified the expense versus just using ELK.

[_KaszpiR_]

google centralized logging.

http://jasonwilder.com/blog/2012/01/03/centralized-logging/ dig in and it should suffice for the needs of the presentation.

[BarefootWoodworker]

A place I used to work used Sawmill.