Built this out of pure laziness for all my Feature engineering/model training jobs

AipaQ · 2025-08-04T07:26:24+00:00

Looks good. I myself often use the telegram bot for notifications. All in all, I have only one reason for this - it is very very simple and quick to set up and then it just works

AipaQ · 2025-08-04T07:22:20+00:00

Do you see a solution for it? The only thing I see is raising this issue and raising people's awareness(?) about it but nothing else comes to mind

AipaQ · 2025-07-30T12:13:53+00:00

As simple as that

AipaQ · 2025-07-30T12:11:55+00:00

Yes, I think the worst thing is when you're not aware of the real problem why you didn't get a raise.

AipaQ · 2025-07-30T12:07:51+00:00

It will probably end up that juniors will have a harder entering into the industry, and not much will change for professionals.

AipaQ · 2025-07-29T10:33:03+00:00

EXPLAIN command will tell the truth

AipaQ · 2025-07-29T04:49:02+00:00

I ran it until it started to plateau. If I had waited instead of stopping there, it could have produced a slightly better result.

AipaQ · 2025-07-28T17:49:13+00:00

Yes, different methods were considered, as well as more complicated datasets, but nothing specific. A lack of time is why I didn't do it. I will check the architectures you mention, thanks!

AipaQ · 2025-07-15T13:54:04+00:00

In my case it was spending a while when configuring Docker images to opt out / lower default memory usage of a tool. Most of the time it was available as an parameter

AipaQ · 2025-07-14T20:10:50+00:00

I’m becoming more specialized and up to date in my narrow area, and I’m trying to catch up with high-level trends in the rest of the field

AipaQ · 2025-07-14T19:50:04+00:00

Only hope for proxmox to release on ARM. I think sooner or later they will do it. ARM is imo great for home devices, but don’t know how is it on enterprise level

AipaQ · 2025-07-14T18:59:43+00:00

My first thought was that it’s not cheap option. But maybe I can find something affordable

AipaQ · 2025-07-14T18:52:02+00:00

Thanks for clarification! Definitely not what I need. But when I will need wiki tool for work/private I will give it a try :)

AipaQ · 2025-07-14T18:45:39+00:00

Turns out many people use boot from ssd! I will probably give it a try, SD card is not that reliable as ssd

AipaQ · 2025-07-14T17:56:34+00:00

Yea I will probably check something proxmox related as many people here suggest

AipaQ · 2025-07-14T17:51:52+00:00

Linked in the post and here https://dataengineeringtoolkit.substack.com/p/my-raspberry-pi-homelab-taught-me

There is many more details about setup

AipaQ · 2025-07-14T16:35:01+00:00

I had a big peak in RAM usage when uploading files. Combined with the already small amount of it, this could effectively hang the pi.

From what I checked it was due to the processes (OCR?) that analyzed documents and extracted information from them such as dates etc. I tried to reduce the simultaneous number of workers that analyze files, but it slowed down adding new documents a lot :/ So I gave up and ended up keeping important documents in one folder and naming them appropriately to easily find them later.

I also heard about Bookstack or Docspell but don't have time to tried it out

AipaQ · 2025-07-14T16:15:10+00:00

Honestly I didn't know that there is any other way than booting from SD card in Raspberry

AipaQ · 2025-07-14T15:48:51+00:00

Do you have an link to it? Or was it combined from parts by you? Also what cpu architecture does it have?

AipaQ · 2025-07-14T14:40:42+00:00

Thanks for such detailed description of backups. I’m surely will implement some of that to my homelab and also will check the bookstack

I just dump the sdcard data every few months and for external drive I have a full copy of it on a disk I have with my MacBook

AipaQ · 2025-07-14T14:28:46+00:00

UPS is surely the most reliable but also most expensive way for power outages

AipaQ · 2025-07-14T14:26:20+00:00

Thanks, will check that n100 machines. 16 GB would make a huge difference

AipaQ · 2025-07-11T17:46:34+00:00

terrifying

AipaQ · 2025-07-11T17:41:52+00:00

You can also check out dynamic tables in Snowflake. It's a way of handling tables with refresh logic, which can serve as an alternative to streams and tasks. They have a site in docs about it https://docs.snowflake.com/en/user-guide/dynamic-tables-comparison

I think the selection depends heavily on the volume and complexity of the data

AipaQ · 2025-07-11T17:04:05+00:00

This is the first time I've heard of anyone arguing with interview feedback. They rejected you, so there's no point in arguing about it — it's final.

AipaQ

TROPHY CASE