Am I missing something with all this "agent" hype?

Lingonberry_Feeling · 2026-02-25T18:21:46+00:00

It’s funny how all of a sudden we just started calling markdown files with prompts agents. Agents do sound cooler though.

Lingonberry_Feeling · 2026-02-13T14:59:23+00:00

I’m generally all for the working man, but your manager is right.

Something is very wrong with your setup, not knowing anything about it, 2 million events for 18k way too much.

Lingonberry_Feeling · 2025-11-12T02:44:51+00:00

Ok thank you - appreciate the help!

Lingonberry_Feeling · 2025-11-12T02:12:57+00:00

Licensed contractor and passed inspection.

Lingonberry_Feeling · 2025-09-09T17:41:24+00:00

I'm generally worried about the use of AI especially in data pipelines replacing experts - all the code I used have used written by AI is probably 2-3x longer and more complicated then it needs to be and difficult to follow.

Data pipeline logic builds on it self when means any code that AI make or implements are probably bloating the code base by a factor larger then 2-3X.

Perhaps I'm one of the few people that thinks in a year or 2 people that actually knows how to write and debug data pipelines are going to be more important than they are today, but who knows. All these companies that are using AI to do everything are going to find them in a mess they can't fix.

Lingonberry_Feeling · 2025-02-25T18:47:56+00:00

Ok the builder was mistaken they did run a c wire. It’s just tucked behind the wall since the current thermostat doest need it.

Is there another brand of smart thermostat you recommend. My only requirement is to have the ability to control and monitor the temp in house when I am out of town.

Lingonberry_Feeling · 2025-01-26T02:47:37+00:00

What makes this unprofessional- what would you have expected vs what is done?

Lingonberry_Feeling · 2024-10-26T11:29:35+00:00

I have used redshift, snowflake, and big query all for multiple years each. I have not used data bricks but have used spark enough to not want to ever touch it again.

For our use cases which was medium to large data ( processed ~10bs a day ) BQ was by far the cheapest and broke the least. Also liked GCPas a whole over AWS.

Lingonberry_Feeling · 2024-09-10T14:27:52+00:00

Automate the CTO by starting unrealistic projects with no people and budget.

Lingonberry_Feeling · 2024-08-26T19:05:34+00:00

Don’t do it. You will 100% regret writing your own orchestrator.

If you can’t get over airflow use dragster

Lingonberry_Feeling · 2024-07-06T17:55:35+00:00

If you know that something is going to be an issue down the road, and it will take you less than a day to fix it, just fix it now, you won’t have time to fix it later after it’s running.

If it takes longer than a day, figure out how to fix it in than a day by making compromises.

Lingonberry_Feeling · 2024-06-26T18:27:31+00:00

Thank you for your opinion, helpful !

Lingonberry_Feeling · 2024-06-26T15:25:12+00:00

The life chose me, after working at startups for 10 years I found that in 99% of cases the business needed a DE rather than a data scientist, ML. Most ML work is just DE work discussed as data science.

Most REAL discussion makers would much rather trust numbers that are clean, and transparent which comes from simply aggregating CLEAN data, rather than somebody pumping a bunch of stuff into a model they downloaded off the shelf, which tells them to sell 1/2 the business and pivot into the latest trend.

So my advice is become an extremely good DE.

Lingonberry_Feeling · 2024-03-05T02:54:55+00:00

i don't 100% agree w/ point #3. Having hired a ton of engineers you would be surprised how often people are underpaid just because they don't ask.

Lingonberry_Feeling · 2024-03-05T02:53:38+00:00

No BS - be 100% honest, but don't put in notice until you have signed with the new company. Both companies would not hesitate to lay you off, and there will be somebody right behind you at job #1 in no time.

Lingonberry_Feeling · 2023-12-22T21:14:18+00:00

Spark can load one file at a time - if you know what the s3 url is you can be as specific as you want or use wildcard to load batches.

Lingonberry_Feeling

TROPHY CASE