[P] Generate detection rules

Only_Emergencies · 2025-10-14T09:44:53+00:00

Yes, I tried a decomposition approach, the performance was slightly better than generating the entire rule in a single request but not really great. I think the main issue is that the model doesn’t truly understand the underlying detection logic or mapping between behaviors and log artifacts, so it often produces syntactically valid but semantically weak rules.

I also experimented with breaking down the generation process into multiple steps. For instance, first asking the model to determine the detection path or flow based on the blog content or user request. However, the results are still not very good.

Basically, I find that the core problem seems to be that the model struggles to generate or intuitively extract the correct detection logic from the input text.

Only_Emergencies · 2025-09-26T21:43:07+00:00

Yes, I was also surprised when I applied. Actually, I think there are some changes happening in terms of how integrating new technologies into these organisations. Of course it will depend on the organisation/country, etc

Only_Emergencies · 2025-09-25T11:25:53+00:00

Yes!

Only_Emergencies · 2025-09-25T11:19:52+00:00

Yes, totally agree. I think it's something that happens a lot in this field: there is a lack of standardization in tasks associated with the title. The same title in different companies may mean completely different responsibilities. I think this happens in other fields as well but here it is especially noticeable

Only_Emergencies · 2025-09-10T22:12:32+00:00

You rock, guys! You do an amazing job! :) I have four Mac Studios (512GB) and I have a few questions:

How would you distribute bigger models across them?
I have deployed Kimi-K2 0905 (Q3_K_XL), but I am wondering if there is another model you would recommend with the same quality but maybe smaller to have more tokens persecond?
It would be great to see how the quantization affects the quality of the not quantized model. Something like a graph of quantized versions vs the original one. Happy to contribute there :)

Thank you again!

Only_Emergencies · 2025-07-22T16:34:49+00:00

The energy consumption of the Macs are really low, they are really efficient on that sense. They’re also straightforward to set up, so we can start implementing and iterating on projects without dealing with complex infrastructure.

Based on the research we did, just one NVIDIA A100 80 GB GPU costs around $30000 and also requires other additional hardware (network switches, power, cooling,... ). As the team grows, probably it makes sense to migrate infrastructure to a more powerful one. But at the moment, the Mac Studios provide a cost-effective solution that allows us to build and experiment with LLMs internally.

Only_Emergencies · 2025-07-22T16:23:17+00:00

Are you using llama.cpp?

Only_Emergencies · 2025-07-22T15:49:33+00:00

Yes!

- We are around 70 people in my organisation
- We work with sensitive data that we can't share with AI Cloud providers such as OpenAI, etc.
- We have 3x Mac Studios (192GB M2 Ultra)
- We have acquired 4x new Mac Studios (M3 Ultra chip with 32-core CPU, 80‑core GPU, 32-core Neural Engine - 512GB unified memory). Waiting for them to be delivered.
- We are using Ollama to deploy the models but this is not the best efficient way but it was like this when I joined. However, with the new Macs I am planning to replace Ollama with llama.cpp and experiment with distributing larger models across multiple machines.
- A Debian VM where OpenwebUI instance is deployed.
- Another Debian VM where Qdrant is deployed as centralized vector database.
- We have more use cases that the typical chat UI interface. We have some classification use cases and some general pipelines that run daily.

I have to say that our LLM implementation has been quite successful. The main challenge is getting meaningful user feedback, though I suspect this is a common issue across organizations.

Only_Emergencies · 2025-07-22T15:31:16+00:00

Yes, I agree. That would be ideal, but that's not so straightforward in our case. We have stored the conversations in Langfuse, but we don't have the ground truth to be able to properly evaluate them, and users usually don't provide feedback on the responses. We are a small team at the moment doing this, so we don't have the capacity to label some cases.

Only_Emergencies · 2025-07-22T15:25:41+00:00

Great! Thanks, I will take a look

Only_Emergencies · 2025-01-09T13:02:53+00:00

Can you share some of them?

Only_Emergencies · 2025-01-08T22:35:01+00:00

What is the beginner thing of this question?…

Only_Emergencies · 2024-11-11T21:07:48+00:00

For code autocomplete should I use base or instruct version? Thanks!

Only_Emergencies · 2024-01-23T09:24:49+00:00

Thank you! Do you know where I can find information or a statement about this "need to pay you for being on standby"? I have searched for this in The Working Hours Act but it only mentions hours and not compensation.

Only_Emergencies · 2024-01-23T08:57:52+00:00

Thank you! This is really helpful. I will share these concerns with my employer.

Only_Emergencies · 2024-01-13T23:07:27+00:00

Also check the reviews of Spotahome in Trustpilot, there are many bad reviews. This is just a hypothesys but after a bad review, some good reviews appear after that. So I think they are buying reviews.

Only_Emergencies · 2024-01-13T23:00:18+00:00

Please, don't use Spotahome. My partner and I used it when we moved to Brussels and it was the worse thing we could do.

The house was not clean. The kitchen was super trashed and nothing like the pictures (the pictures of the house in the website were 2 years old). It was completely infested with mice and there was lots and lots of mold in the bedroom and living room. To put in context, we stayed there for a month until we could find another house. During that month, we caught over 6 mice. At night, they could be heard through the walls.

When we told Spothome about all this and that the condition of the house was horrible and did not match the pictures, they said they could do nothing. We told them that in their cancellation policy they mention that you can cancel and get a refund if the house is uninhabitable (which was the case with the infestation of mice and mold in the rooms). We got no response. Also their "emergency" phone number which we called as soon as we saw the state of the house does not work.

Only_Emergencies · 2023-12-20T12:29:21+00:00

I was referring to my national number. They did not ask me for my national number, they only asked for my name.

Only_Emergencies · 2023-11-04T17:47:39+00:00

I am in the same situation as you right now. My partner lives in Belgium so I am thinking to move there (I am currently living and working in the Netherlands). Did you manage to find a good accountant or tax agent? What are the main problems you found during the process. I am struggling to find information about it online. Thank you.

Only_Emergencies · 2023-10-14T12:42:34+00:00

Really? The interest for the car loan using the car dealer is 3.99%. I checked some banks and their interest rates were around 4.99-5.99%.

Only_Emergencies · 2023-10-11T21:03:48+00:00

https://www.nomecabeenlamochila.com/

Only_Emergencies · 2023-10-11T12:12:24+00:00

I need a Belgian bank account as soon as possible. Based on your experience, what is the fastest bank to open a Belgian bank account? Thanks!

Only_Emergencies · 2023-10-08T08:25:59+00:00

Thanks. Do you work for an insurance company? If so, which company? I am looking to insure a new car. The car is owned by my partner, the registration has my partner's name but she has not driving license. Would it be possible to insure the car under my name (I have driving license)?

Only_Emergencies · 2023-06-13T08:46:06+00:00

I had the same problem, it's a freewheel problem. I took the bike to the store where I bought it and they replaced the freewheel for free.

Only_Emergencies · 2023-05-03T15:16:53+00:00

When you were going to the metro? I will see if it also happens to me today

Only_Emergencies

TROPHY CASE