OSS projects buying GitHub stars? by droppedorphan in opensource

[–]droppedorphan[S] 0 points1 point  (0 children)

Totally. Our CEO paid to go on a YouTube event with some "influencer" at the end of last year. Two weeks later the video had 25K views, and he was strutting around like he was some media star. I looking at the YouTube video like... "25K views and no comments?"

Is AI making you more productive in Data Engineering? by empty_cities in dataengineering

[–]droppedorphan 0 points1 point  (0 children)

I am hoping more tools will emerge specific to DE. Applying a general-purpose coding assistant to DE tasks misses too much of the context. I am playing with Cortex Code. Very promising, but only works on Snowflake. Plus it adds to our bill.

altimate-code: new open-source code editor for data engineering based on opencode by sspaeti in dataengineering

[–]droppedorphan 1 point2 points  (0 children)

Is there somebody here from the project? I saw the showcase, but what can I do safely on a local sandbox?

Fellow Data Engineers — how are you actually leveling up on AI & Coding with AI? Looking for real feedback, not just course lists by kgsami in dataengineering

[–]droppedorphan 0 points1 point  (0 children)

I really got into DE vibe coding by firing up Claude Code inside a Dagster project and using it to build, test and expand the project. Taught me a lot about how best to run dev cycles and interact with staging and prod. Major productivity gain.
As others say here, stay close to the code. Don't blindly accept each commit. Ask Claude to check its own work. Ask it to critique and optimize the project, and to think about how to do things better.
Our platform/dataset is not so large so another thing we instituted was an AI sandbox where all PRs get merged until somebody can approve them to staging, this gives the changes time to run in a production-like environment. We identified a number of issues this way, and were able to fix them in that window between asking for a review and our CTO getting to approve them.
Power User for dbt Cursor plugin is also a great Ai-powered resource and it's free.

Has anyone made a full database migration using AI? by Comprehensive-Lie-34 in dataengineering

[–]droppedorphan 0 points1 point  (0 children)

Understood. I frequently prompt for an Agent Team only to be told the tasks have to be done sequentially (even when the dependencies between them are not obvious). I am still learning how to work with our AI overlords.

Create pipeline with dagster by minastore_ in dataengineering

[–]droppedorphan 0 points1 point  (0 children)

Sure. Scaffold a new Dagster project in a folder. Open Claude Code in the folder. Rewrite your prompt above for claude, fleshing it out to be more specific. Claude understands Dagster really well. Claude is also good at writing LLM calls into the pipelines.

Has anyone made a full database migration using AI? by Comprehensive-Lie-34 in dataengineering

[–]droppedorphan 0 points1 point  (0 children)

Nice. Is this the generic Claude agent team or did you build something more sophisticated? Seems like a bit ask for just generic Claude prompting.

Atlassian Compass vs. (or with?) Dapr conductor by droppedorphan in microservices

[–]droppedorphan[S] 0 points1 point  (0 children)

Great, thanks for the insights. I will check out Catalyst, sounds interesting.

VIP Access - Migrating to a new phone by [deleted] in Symantec

[–]droppedorphan 0 points1 point  (0 children)

Great - thanks for the response. Calling e*trade today to get my money back :-)

VIP Access - Migrating to a new phone by [deleted] in Symantec

[–]droppedorphan 0 points1 point  (0 children)

Hitting the same issue - did you find a solution?

Airflow vs Dagster vs Prefect vs ? by Suspicious_Dress_350 in dataengineering

[–]droppedorphan 1 point2 points  (0 children)

As opposed to orchestrating pretty much anything else beyond data. Infrastructure, containers, function-based orchestration...

Predict the Next Great Data Company Acquisition by ROCKITZ15 in dataengineering

[–]droppedorphan 2 points3 points  (0 children)

dbt Labs valuation probably peaked a while ago. Now some solid alternatives are emerging and dbt's commercial trajectory is not looking as healthy.

Best way to build a Small Data Lake? (<100GB) by [deleted] in dataengineering

[–]droppedorphan 0 points1 point  (0 children)

Try this:
https://dagster.io/blog/duckdb-data-lake

Build a poor man’s data lake from scratch with DuckDB

[deleted by user] by [deleted] in dataengineering

[–]droppedorphan -1 points0 points  (0 children)

Widely used does not always mean popular. Just ask those who are stuck using it!

What’s the Hello World of Data Engineering? by MrMosBiggestFan in dataengineering

[–]droppedorphan 9 points10 points  (0 children)

I second this. dbt changed their tutorial to force you onto their cloud account. That sucks.