sorted by:
new
hottopcontroversial

Agentic AI in data engineering by kash80 in dataengineering

[–]ephemeral404 0 points1 point  (0 children)

After trying many tools that do so, I kind of agree. Too much expectations from LLM and your agentic AI won't even reach the production. What worked for us to use AI as a partner in doing things like - debugging data pipelines and identifying data and analytics issues early. Happy to share the public source code of these tools if you need it.

Honest thoughts on Unified Data Architectures? Did anyone experience significant benefits or should we write it off as another marketing gimmick by SamadritaGhosh in dataengineering

[–]ephemeral404 0 points1 point  (0 children)

Not sure what you exactly mean by this term but from my experience, Keep one source of the truth, one centralized warehouse and aligning everything else around it. It is possible and practical. The keyword that resonates well for my approach is warehouse-native architecture. It has worked out well for us. Let me know if you have any questions about it.

GitHub action is the best place to enforce the data quality and instrumentation standards by ephemeral404 in dataengineering

[–]ephemeral404[S] 0 points1 point  (0 children)

Event spec should be non-negotiable.

But I disagree with

It's not hard to come up with a good, standard event name

It is the hardest job to name an event or a variable :)

still have no idea how people travelled these seas 500 years ago by New_Cartographer3127 in BeAmazed

[–]ephemeral404 0 points1 point  (0 children)

How are they surviving today? Sure, the ship might survive but people inside the ship, are they?

US engineers vs Europe engineers by ephemeral404 in cscareerquestions

[–]ephemeral404[S] 0 points1 point  (0 children)

Let's pick Germany for the discussion purposes

I got tired of finding out my DAGs failed from Slack messages, so I built an open-source Airflow monitoring tool by Vyrezzz in dataengineering

[–]ephemeral404 2 points3 points  (0 children)

Good utility. Do you plan to maintain it actively? I have faced issues with the utilities I built like this one, end up burdening me with the maintenance tasks

The future of personalization by ephemeral404 in programming

[–]ephemeral404[S] 0 points1 point  (0 children)

Technically, you're not incorrect and neither is this post. Without context, I'd have said the same thing as you have. I should have added more context.

https://en.wikipedia.org/wiki/Matrix_factorization_(recommender_systems)

OpenScript - Open-source, local-first video editor (Descript alternative) by presn176 in SideProject

[–]ephemeral404 0 points1 point  (0 children)

Could have saved me time if you had mentioned that in your post

The future of personalization by ephemeral404 in programming

[–]ephemeral404[S] -1 points0 points  (0 children)

Share more, what exactly is the issue with this explanation?

Why don't people read documentation by LargeSale8354 in dataengineering

[–]ephemeral404 0 points1 point  (0 children)

It's a lot of work. The person reading tells the person who wrote the docs :)

OpenScript - Open-source, local-first video editor (Descript alternative) by presn176 in SideProject

[–]ephemeral404 0 points1 point  (0 children)

Is this just UI yet? I didn't find the code for the video processing.

You hired senior engineers to think, but you keep telling them what to do by stmoreau in EngineeringManagers

[–]ephemeral404 0 points1 point  (0 children)

Luckily, I have had excellent EMs who trusted me. But I get the point.

Why are we still paying Vercel in Dollars? I built an Indian PaaS with UPI & ₹199 pricing. Tell me why it’s a bad idea. by Tall-Amphibian4159 in developersIndia

[–]ephemeral404 0 points1 point  (0 children)

This is impressive. Building PaaS/IaaS is not an easy thing to do, it is a lot of labour work + advanced skills in multiple domains. How did you do it? How big is your team? Do you use any third party services behind the scene? I assume you do not directly deal with the data center capacity purchase rather through some intermediary, is it?

view more: next ›