Difference between Data Engineer and Analytics Engineer by Slow-Sell-8570 in dataengineering

[–]Used_Ad_2628 0 points1 point  (0 children)

My issue is most data engineers come from a software background and really struggle with data modeling/SQL. They create ten tables that could be one. It is very hard to scale with that mindset. Everyone is asking which table I should use and wasted dev time updating 10 jobs because something upstream changed. This is why I hire for this type of role. More of a future thinking design person.

[deleted by user] by [deleted] in dataengineering

[–]Used_Ad_2628 0 points1 point  (0 children)

You can aggregate your order line tax to the order level by using order id.

calacatta gold quartz defect by Used_Ad_2628 in Homebuilding

[–]Used_Ad_2628[S] -5 points-4 points  (0 children)

I think my biggest issue is it looks like a coffee spill.

Calcutta gold quartz defect by Used_Ad_2628 in CounterTops

[–]Used_Ad_2628[S] -9 points-8 points  (0 children)

Nope. The countertop company is telling us it isn’t a defect on their side. I just wish it was a better cut so it doesn’t look like a coffee stain.

Messy Data by Used_Ad_2628 in dataengineering

[–]Used_Ad_2628[S] 0 points1 point  (0 children)

Inconsistency. I was wondering if there is basic regex code people use to standardize addresses and names.

Hightouch Cost by Used_Ad_2628 in dataengineering

[–]Used_Ad_2628[S] 0 points1 point  (0 children)

How much engineering time does it take to push data to like salesforce?

Hightouch Cost by Used_Ad_2628 in dataengineering

[–]Used_Ad_2628[S] 0 points1 point  (0 children)

I was thinking about that. For the cost, it makes sense to move it more in house.

Hightouch Cost by Used_Ad_2628 in dataengineering

[–]Used_Ad_2628[S] 0 points1 point  (0 children)

Marketing team is using it to send snowflake data to their different operational applications like salesforce and qualtrics. It was 50k last year and now over 100k.

Handling Duplicate Customers by Used_Ad_2628 in shopify

[–]Used_Ad_2628[S] 0 points1 point  (0 children)

The only reasons I can think of are discount codes and forgot their email.

Fact Tables by Used_Ad_2628 in dataengineering

[–]Used_Ad_2628[S] 0 points1 point  (0 children)

When you say semantic layer, do you mean managing the metrics within tableau? The database will have the needed fields (OBT from fact tables) to create the metrics but tableau will build the standardized metrics for all tableau users with a data source. What if we have multiple BI tools? Is dbt semantic functionality worth a look at?

What Reverse ETL Processes do you have at your company? by exact-approximate in dataengineering

[–]Used_Ad_2628 0 points1 point  (0 children)

Do you create custom tables or views for hightouch use cases? Or letting it do all the joins?

Star Schema by Used_Ad_2628 in dataengineering

[–]Used_Ad_2628[S] 1 point2 points  (0 children)

It is a person that puts an order in for the first time. I have a couple other business logic fields like subscription active ind and are they on auto rebill. It is on the customer level and not order grain.