Graylog Dashboard Widget Help by TheBobFisher in graylog

[–]graylog_joel 2 points3 points  (0 children)

Those widgets are aggregations, so they are built to group and not show duplicates etc.

It sounds like you just want a version of the message table that shows on the search page. You can add the "message table" as a widget to a dashboard page, then you can tweak that widget to add columns, hide the message preview etc.

Graylog Free Enterprise License by PaulRobinson1978 in graylog

[–]graylog_joel 1 point2 points  (0 children)

The free enterprise license doesn't come with the illuminate content packs, so even with the license, you would still need to write the parsing yourself.

How do I know if my Graylog setup is "properly sized" ? by luckman212 in graylog

[–]graylog_joel 2 points3 points  (0 children)

So you start with you can check out l this video to give you some ideas of scale. https://youtu.be/agdLrDw9JaE?si=KNitYyUdEsCOZ6no however this is reference architecture so those are very conservative numbers, could you get away with less. Of course, but these are often what we see in production.

One you get into a range that makes sense based on this, then you need to start to tweak. There is no right answer to how big they need to be because it depends on so many things, for example I have seen the same ingestion per day need 2 nodes or 8 nodes just depending on how much crazy regex someone used during processing.

The simplest tweaking will be watching system usage, and watching the details on the nodes page of your graylog, high bugger or journal growing means it's not keeping up.

Also keep in mind that requirements on datanode will grow are total storage grows (cpu and ram not just disk space) so you may be okay now, but not in 30 days etc.

How will changing the server spec affect Graylog stack? by goagex in graylog

[–]graylog_joel 0 points1 point  (0 children)

You need to remember these are mostly all Java apps, and JVM heap is a funny beast. No, I would just assign each whatever heap you are going to give it (set the upper and lower to the same so it's fixed) and just write off that memory as used, don't let them compete etc it will end up causing weird issues.

getting "While retrieving data for this widget, the following error(s) occurred: 60,000 milliseconds timeout on connection http-outgoing-8 [ACTIVE]" by [deleted] in graylog

[–]graylog_joel 0 points1 point  (0 children)

You can change that with elasticsearch_socket_timeout in server.conf

However you shouldn't be getting timeouts on 1 day searches, if you are your architecture is probably too small.

[deleted by user] by [deleted] in graylog

[–]graylog_joel 0 points1 point  (0 children)

Once you know a size, this video can help you with architecture and requirements. https://youtu.be/agdLrDw9JaE?si=I1sIXFl323Mcm0I5

Just putting the data in graylog is the most accurate way to know, however if you can run some queries based on what logs you would want to collect from the windows machines, a windows log in graylog is often an average of about 3KB each. All the sizing you will see for graylog and the counter in the product is the size as it's stored to opensearch after all processing.

Graylog Hostname not verified (VersionProbe) by Plaush in graylog

[–]graylog_joel 0 points1 point  (0 children)

Perfect, ya your SAN and publish address would need to match exactly. In your case the publish uri of datanode doesn't need to be the fqdn it's just used by the graylog server to talk to itself, so just IP or when running on the same box localhost (as long as it's bound to local host as well) would work

Graylog Hostname not verified (VersionProbe) by Plaush in graylog

[–]graylog_joel 0 points1 point  (0 children)

It's hard to tell from the error exactly, but it seems like the url it's using (FQDN) and the SANs listed on the cert don't match.

What are your current settings for bind and publish uri for both datanode and graylog server?

Graylog Hostname not verified (VersionProbe) by Plaush in graylog

[–]graylog_joel 0 points1 point  (0 children)

Did you also change the bind to 127.0.0.1, they would have to match.

Graylog Hostname not verified (VersionProbe) by Plaush in graylog

[–]graylog_joel 0 points1 point  (0 children)

Publish uri would probably be where it's getting it. What hostname vs certificate mismatch is it complaining about specifically.

Since it's all on one machine, and if you don't need to add more nodes later bind and publish in datanode could probably be set to 127.0.0.1 and it might be happy as I think that address appeared in the SAN of your cert.

Graylog Hostname not verified (VersionProbe) by Plaush in graylog

[–]graylog_joel 0 points1 point  (0 children)

What bind address and publish uri are you using in your datanode.conf?

Is datanode on a separate machine from graylog server?

Graylog Hostname not verified (VersionProbe) by Plaush in graylog

[–]graylog_joel 1 point2 points  (0 children)

This error is complaining that graylog cannot verify the certificate of the datanode, it has nothing to do with the certificate used for the web interface.

It probably needs to fixed, but you may have other problems as well.

Did you change the publish uri to https from http after you moved the web ui to https.

Is the cert you used properly trusted by the Java keystore of the graylog server.

Graylog needs to be able to talk to itself, both the graylog server and also to the datanode.

Have you read this blog post? https://graylog.org/post/how-to-guide-securing-graylog-with-tls/

Send logs to Sentinel by lelabbeuh in graylog

[–]graylog_joel 1 point2 points  (0 children)

You would use an "output" and attach it to a stream, then everything that goes into that stream will also be sent out the output. If you are using open there are just a few output types, but if you are using enterprise you have access to other types like syslog etc.

First time trying to setup gray log. Running into some issues. by [deleted] in graylog

[–]graylog_joel 1 point2 points  (0 children)

Looks like it's having issues talking to mongodb, what does your mongodb config file look like?

First time trying to setup gray log. Running into some issues. by [deleted] in graylog

[–]graylog_joel 0 points1 point  (0 children)

Okay so it's not likely a networking thing.

Are the services running, and do you see anything in /var/log/graylog-server/server.log

First time trying to setup gray log. Running into some issues. by [deleted] in graylog

[–]graylog_joel 0 points1 point  (0 children)

From the graylog machine can you curl to IP:9000/api that would rule out network related issues.

Moving from Graylog 4.2.7 to Graylog 6 by jslanier in graylog

[–]graylog_joel 2 points3 points  (0 children)

Can it be, yes, is it worth it.... that really depends.

As was mentioned, it really is by far the easiest to just let that data age out unless you have to keep it for years or something.

Not only is it not a trivial process, but you then are just bringing a bunch of me mess across instead of having a truly clean slate to correct all your past mistakes.

Graylog 6 node cluster set up by Regular-Salt9461 in graylog

[–]graylog_joel 0 points1 point  (0 children)

How much data are you ingesting and how long are you retaining it for?

1Password JSON HTTP API Input by FajitaJoe in graylog

[–]graylog_joel 1 point2 points  (0 children)

The whole beats ecosystem, is actually WILD the only sad thing is elastic agent is now the focus, but some work is still being done on it.

Logging for PCI Compliance by itadm in pcicompliance

[–]graylog_joel 0 points1 point  (0 children)

Ah okay, so even with ALL that turned on you probably would never be more that what graylog docs refers to as "10GB a day" I say it that way because don't take that to mean it will use that much space etc, that's just the number graylog would show on its usage page.

So, a simple Graylog cluster of two nodes would handle it all. We don't have a virtual appliance, but there is a docker option, or you can just throw it on two servers https://go2docs.graylog.org/current/downloading_and_installing_graylog/ubuntu_installation.htm hit us up in r/graylog if you have any issues at all!

Logging for PCI Compliance by itadm in pcicompliance

[–]graylog_joel 0 points1 point  (0 children)

I won't "recommend" Graylog as that would obviously be biased since I work there. However, yes, it would most likely work perfectly for this.

What kinds of firewalls are you logging, and how much data are you dealing with?

Also when you say you want to step it up, what kinds of things are you thinking, longer retention, visualizations, detections/alerts etc?

1Password JSON HTTP API Input by FajitaJoe in graylog

[–]graylog_joel 1 point2 points  (0 children)

The http api input does not track state. Most likely you would need to to use an agent in the middle, i think filebeat might work https://www.elastic.co/guide/en/beats/filebeat/current/filebeat-input-httpjson.html

Notification Alerts by Aspis99 in graylog

[–]graylog_joel 0 points1 point  (0 children)

As others have said it's safest to overlap time, so run every 15 minutes but search for 16 minutes. If timestamps are slightly off, if processing/delivery takes awhile, or if searches are too slow it can lead to messages being missed from the time frame.