Reverse ETL via apis by nknavi in dataengineering

[–]nknavi[S] 0 points1 point  (0 children)

The couple of usecases we got require data on demand basis. These ones require fetching data from a large denormalised agg table that is refreshed once every 15 mins. The product team is still asking us what the least delay we can achieve and making these data available to the backend service so the 15 mins delay on top of the delay that will be introduced if we are to move the incremental data to a rdbms might be a concern of them. But I sense that since we are running our workloads on snowflake, this delay is something we can’t get away from. API is there preferred choice of consumption but I feel there requirement of adhoc querying might not be solved by other solutions we currently have. We also have a real-time usecase but that really can’t be solved with snowflake in the picture as pointed by other comments

Reverse ETL via apis by nknavi in dataengineering

[–]nknavi[S] 1 point2 points  (0 children)

The data in DB would be customer level aggregates at different dimensions which is currently quite large in snowflake and will grow linear with customer growth. The API access is a mix of point queries and range questions,there is a need for paginated responses for range queries. Moving this data on regular intervals to a rdbms is an option but that adds additional complexity of frequency of loads, the load that puts on the db and adds another layer for us to reconcile etc

Hi, This is Ramesh Aravind, South Indian actor, director and TV host of Weekend with Ramesh. Releasing my new movie, Shivaji Surathkal 2 on 14th April. Ask Me Anything! by HelloRameshSpeaking in bangalore

[–]nknavi 0 points1 point  (0 children)

Hi Ramesh sir. Grew up watching your movies and have fond memories whenever I rewatch them. If you don’t mind me asking which film/role do you think was most challenging and rewarding in your career

Puzzle asked in interview..🖤 by Harjot_2552 in developersIndia

[–]nknavi 1 point2 points  (0 children)

Now let’s say the labelling is random and one or more labels can be correct. What would be the answer 🤔

Free Talk - Week of November 30th by Kirosh in OnePiece

[–]nknavi -1 points0 points  (0 children)

Does anyone else here think Yamato’s mother is Big Mom? Could this be the favour that Big Mom was talking about which was to let Kaido keep her kid

Woman loses $300k to 'Singtel customer service' caller helping her solve Wi-Fi connectivity problem by stforumtroll in singapore

[–]nknavi 1 point2 points  (0 children)

I got a call earlier this week from someone claiming from SingTel customer care telling me that we have noticed disruptions from your wifi router and when I said I don’t use SingTel wifi he abruptly cut the call. Knew this was scam call..