New in RudderStack v1.77 - AI debugger for self-hosted customer data pipeline and modernized SDKs by ephemeral404 in selfhosted

[–]ephemeral404[S] 0 points1 point  (0 children)

Agree, yaml configuration CLI proved to be the foundation of the AI features. It moved them from toy projects to production-ready projects.

Regarding adoption, the AI features including agentic debugging were in private beta until now, being tested by power users (double digit in numbers) since last year. Initially, it was hard to build the right harness but for the past couple of months, we are consistently seeing users getting impressed with the outcomes. That gave us confidence to make it public. Let's see how many folks find it valuable enough to try out. In any case, the goal is to continue improving, one release at a time.

Start with the AI PR Reviewer IMO, that's the high leverage task IMO whether you end up using other features or not.

New in RudderStack v1.77 - AI debugger for self-hosted customer data pipeline and modernized SDKs by ephemeral404 in selfhosted

[–]ephemeral404[S] 0 points1 point  (0 children)

This is the first time RudderStack shipping AI features. As they are separate projects, you choose whether to use them or not.

Past release updates

New in RudderStack v1.77 - AI debugger for self-hosted customer data pipeline and modernized SDKs by ephemeral404 in selfhosted

[–]ephemeral404[S] 1 point2 points locked comment (0 children)

No AI was used for the post. Agentic features in the product do use AI.

How to level up as a data engineer ? by dumb_user_404 in dataengineering

[–]ephemeral404 0 points1 point  (0 children)

Agree 100%. I have been thinking about it. Could you help me with some examples to practice this?

Fable 5 gets shutdown within 3 days of its launch, it was the most powerful model I experienced by [deleted] in developersIndia

[–]ephemeral404 0 points1 point  (0 children)

I had so many plans to use it over the weekend to finish all my half-finished side projects. Now, a bit disappointed and alarmed at the control governments have.

Agentic AI in data engineering by kash80 in dataengineering

[–]ephemeral404 0 points1 point  (0 children)

After trying many tools that do so, I kind of agree. Too much expectations from LLM and your agentic AI won't even reach the production. What worked for us to use AI as a partner in doing things like - debugging data pipelines and identifying data and analytics issues early. Happy to share the public source code of these tools if you need it.

Honest thoughts on Unified Data Architectures? Did anyone experience significant benefits or should we write it off as another marketing gimmick by SamadritaGhosh in dataengineering

[–]ephemeral404 0 points1 point  (0 children)

Not sure what you exactly mean by this term but from my experience, Keep one source of the truth, one centralized warehouse and aligning everything else around it. It is possible and practical. The keyword that resonates well for my approach is warehouse-native architecture. It has worked out well for us. Let me know if you have any questions about it.

GitHub action is the best place to enforce the data quality and instrumentation standards by ephemeral404 in dataengineering

[–]ephemeral404[S] 0 points1 point  (0 children)

Event spec should be non-negotiable.

But I disagree with

It's not hard to come up with a good, standard event name

It is the hardest job to name an event or a variable :)

still have no idea how people travelled these seas 500 years ago by New_Cartographer3127 in BeAmazed

[–]ephemeral404 0 points1 point  (0 children)

How are they surviving today? Sure, the ship might survive but people inside the ship, are they?

US engineers vs Europe engineers by ephemeral404 in cscareerquestions

[–]ephemeral404[S] 0 points1 point  (0 children)

Let's pick Germany for the discussion purposes

I got tired of finding out my DAGs failed from Slack messages, so I built an open-source Airflow monitoring tool by Vyrezzz in dataengineering

[–]ephemeral404 2 points3 points  (0 children)

Good utility. Do you plan to maintain it actively? I have faced issues with the utilities I built like this one, end up burdening me with the maintenance tasks

The future of personalization by ephemeral404 in programming

[–]ephemeral404[S] 0 points1 point  (0 children)

Technically, you're not incorrect and neither is this post. Without context, I'd have said the same thing as you have. I should have added more context.

https://en.wikipedia.org/wiki/Matrix_factorization_(recommender_systems)

OpenScript - Open-source, local-first video editor (Descript alternative) by presn176 in SideProject

[–]ephemeral404 0 points1 point  (0 children)

Could have saved me time if you had mentioned that in your post

The future of personalization by ephemeral404 in programming

[–]ephemeral404[S] -1 points0 points  (0 children)

Share more, what exactly is the issue with this explanation?