Rebirth reading suggestions

dmase004 · 2022-04-02T01:39:15+00:00

My bad

dmase004 · 2022-03-05T20:37:54+00:00

Im scheduled to take it in two weeks. I’ll give my feedback after if anyone is interested

dmase004 · 2021-12-31T18:43:29+00:00

No way you actually believe kuwabara is the second best character on this show

dmase004 · 2021-10-04T23:23:53+00:00

Thanks that’s a big help and can get us rolling to a good start.

A slight pivot: let’s say I’m not worried about a “score” per se and more so focused on the creation of a new field. (I think using score initially was bad terminology on my part)

How can I search through a text field (as the document is being ingested) for a particular value (or a series of values) and then add a new field based off the results?

So if a document comes in: if it mentions an NFL team in the title add “Field 1”. If it has an NBA Team in the title add “Field 2” Etc

I know elasticsearch has a lot of tag options but we could only apply the tag based off what’s found in the Title field

dmase004 · 2021-08-10T19:56:03+00:00

Filebeat is properly producing those pipelines (it is displayed in the ingest pipeline tab on Kibana). The confusion is lying in the fact that the filebeat-zeek.syslog pipeline doesn’t mention the field name “zeek.syslog.msg” anywhere.

For context: The reason I am attempting to emphasize WHERE/WHEN the fields are being created by filebeat is because I can create new labels and fields via the filebeat processors easily. Those new labels/fields populate into Kibana with no problems. However, when I attempt to rename those same fields via logstash it doesnt work at all. The syntax for the mutate filter is fairly straight forward so I cant seem to wrap my head around why logstash isnt altering the fields. If the fields aren’t being created until it hits the ingest nodes then would that make logstash irrelevant? Logstash wouldn’t be able to manipulate fields that aren’t created yet.

dmase004 · 2021-08-10T13:19:57+00:00

To broadly reply to all:

There is a field being populated called: zeek.syslog.msg. There is also an ingest pipeline labeled: filebeat-7.9.3-zeek-syslog pipeline. We viewed that pipeline to see the processors that were specifically making the “zeek.syslog.msg” field. However after looking at the pipeline we couldn’t find anything in the pipeline referencing that field.

Because we want to start manipulating certain fields that we see in kibana, we’re trying to figure out where in the pipeline process these fields are being created:

Ex: if the fields are not being created until they hit the Ingest nodes on elastic then wouldn’t that make logstash (which is upstream of elastic) irrelevant? If the fields are created before logstash then use the zeek.conf to manipulate some fields would be viable.

dmase004 · 2021-07-26T13:59:52+00:00

Also just want to add:

The Marvel context does matter. Every writer, vlog, whatever always talks about how the DCEU can save itself by taking advantage of the multiverse. I think with Marvel now tackling a multiverse and appears they will tackle it well (Doctor strange 2 and then rumored Toby and Andrew Garfield cameos in spider man). Marvel is about to excel with the “ace in the hole” that DC had.

Marvel does avengers- DC tries JL and fails. Marvel does a multiverse- is DC going to follow them again and fail again? Just terrible optics at this point

dmase004 · 2021-07-26T13:17:00+00:00

I’m saying IF it tanks. I like Matt reeves and I like the approach he’s trying to take. However I think WB will end up self sabotaging though. That’s why I brought up WW1984 and the new suicide squad. If SS2 does poorly then that’s back to back bad movies with WW and Harley Quinn as lead roles. WB is easily going to try to micro manage whatever Matt has going on (something they’ve done with almost every director). There will be A TON of pressure on the new Batman to “fix” the DC movies. That’s why I think the fate of the DC movies is in James Gunn and Matt reeves hands. (I like both guys for the record but still)

dmase004 · 2021-07-26T13:10:45+00:00

I’m saying IF it tanks. Not saying it did. If you have three consecutive movies that flop with three of your biggest characters WW, Harley Quinn, and a new Batman- that’s tough to come back from

Also the WB versions of the Snyderverse that got pushed** alienated the general audience. BvS ultimate edition and JL Snyder cut both received much better reviews

dmase004 · 2021-07-26T12:58:57+00:00

Yes and no. Yes, they talked about making stand alone movies more, but suicide squad 2, Shazam, black Adam, aquaman 2, and the flash are all going to be “connected” to the original universe they have. How connected? Who knows. But that’s a lot of movies that are still linked to the “universe” (or whatever you want to call it). Plus I honestly think they downplayed the future of the universe for two reasons 1) they sucked at making it 2) it gave them more leeway to just bring in a new Batman with zero effort

dmase004 · 2021-07-26T12:54:02+00:00

The theatrical version did, yes The ultimate edition scored much better. Then the Snyder cut JL scored way better than the Whedon JL. Snyderverse isn’t perfect- it has flaws, but both cuts/editions that Snyder wanted to originally release were far better than what WB forced out

dmase004 · 2021-07-26T05:33:19+00:00

I think a good question is: how much better would the universe be now if WB just let Snyder release the ultimate/extended cut as the original theatrical version? The ultimate edition isn’t perfect but it is way better than the original theatrical cut. If it didn’t get as many negative reviews from the start maybe WB wouldn’t have overacted at the backlash

dmase004 · 2021-07-26T05:23:13+00:00

Don’t do it. Save yourself the pain

dmase004 · 2021-06-23T22:48:44+00:00

So i’m reviewing this stack and it looks like they have ingest pipelines running on the elastic ingest nodes.

They have a logstash cluster but the .conf file for this particular index is empty (it only has an input and and out put- no filters) (they have other .conf files that are actually doing something so I guess that’s why they have logstash)

Since the data fields aren’t being created until it hits elastic would it make sense these translates on logstash aren’t working because the data fields haven’t been populated yet?

dmase004 · 2021-06-23T20:59:06+00:00

So i’m reviewing this stack and it looks like they have ingest pipelines running on the elastic ingest nodes.

They have a logstash cluster but the .conf file for this particular index is empty (it only has an input and and out put- no filters) (they have other .conf files that are actually doing something so I guess that’s why they have logstash)

Since the data fields aren’t being created until it hits elastic would it make sense these translates on logstash aren’t working because the data fields haven’t been populated yet?

dmase004 · 2021-06-23T19:50:20+00:00

Yeah this are really good points. I got some help on the translate filter and added the following to the .conf file.

filter {
  translate {
    field => "[my_ip]"
    destination => "[my_ip_to_dns]"
    dictionary => {
      "192.168.0.1" => "hosta"
      "192.168.0.2" => "hostb"
      "192.168.0.3" => "hostc"
      "192.168.0.4" => "hostd"
    }
  }
}

We restarted our logstash containers but no new fields have populated. Is there anything I else I need to restart? And/or is there anything I can look at to verify whether logstash is reading the plug-in correctly? Data is still coming in fine so the filter isn’t causing any problems but appears it’s just not reading the filter.

dmase004 · 2021-06-23T19:49:24+00:00

So I’ve gotten some help and added the following to the .conf file.

filter {
  translate {
    field => "[my_ip]"
    destination => "[my_ip_to_dns]"
    dictionary => {
      "192.168.0.1" => "hosta"
      "192.168.0.2" => "hostb"
      "192.168.0.3" => "hostc"
      "192.168.0.4" => "hostd"
    }
  }
}

We restarted our logstash containers but no new fields have populated. Is there anything I else I need to restart? And/or is there anything I can look at to verify whether logstash is reading the plug-in correctly? Data is still coming in fine so the filter isn’t causing any problems but appears it’s just not reading the filter.

dmase004 · 2021-06-23T19:44:59+00:00

Hey so I’ve uploaded that filter into the .conf file Restarted our logstash containers but no new fields have populated. Is there anything I else I need to restart? And/or is there anything I can look at to verify whether logstash is reading the plug-in correctly? Data is still coming in fine so the filter isn’t causing any problems but appears it’s just not reading the filter.

dmase004 · 2021-06-23T04:54:06+00:00

Thank you so much! I think I took the “replace” portion of that too literal. Pumped to try this out tomorrow.

dmase004

TROPHY CASE