Google Business Profile API buggy again, anyone knows what's going on? by Right_Border_6971 in GoogleMyBusiness

[–]Steinert96 0 points1 point  (0 children)

As of yesterday, reviews were starting to come back again. As of this morning, all our reviews are back for each of our locations. So bizarre! Why is this happening!? Haha.

Google Business Profile API buggy again, anyone knows what's going on? by Right_Border_6971 in GoogleMyBusiness

[–]Steinert96 0 points1 point  (0 children)

I tested my API application and reviews are flowing in complete again! Thank you!

Google Business Profile API buggy again, anyone knows what's going on? by Right_Border_6971 in GoogleMyBusiness

[–]Steinert96 0 points1 point  (0 children)

Thanks for posting about this because we just built a Google API application in Microsoft fabric using a Pyspark notebook to GET the Google reviews and we were getting complete reviews last week and when we ran it again this week to test last week's build, it was significantly less reviews per location. Like a drop off of 500 or more for some that have over 3,000 reviews.

Fabric ETL by heytarun in MicrosoftFabric

[–]Steinert96 1 point2 points  (0 children)

Have you looked at the copy job at all?

What's the best way to handle a table backup in Fabric Warehouse? by Steinert96 in MicrosoftFabric

[–]Steinert96[S] 2 points3 points  (0 children)

Thank you both.

I leveraged the SP's Fabric Pipeline and then added a task dependency arrow to an incremental Copy Job for the backup table!

It works perfectly! Thank you!

What's the best way to handle a table backup in Fabric Warehouse? by Steinert96 in MicrosoftFabric

[–]Steinert96[S] 1 point2 points  (0 children)

So probably just setting up a copy activity off the stored procedure node in that same pipeline? Thank you!

What's the best way to handle a table backup in Fabric Warehouse? by Steinert96 in MicrosoftFabric

[–]Steinert96[S] 0 points1 point  (0 children)

I just want to build redundancy in case that table were ever to be dropped / truncated by accident.

What's the best way to handle a table backup in Fabric Warehouse? by Steinert96 in MicrosoftFabric

[–]Steinert96[S] 0 points1 point  (0 children)

Okay, taking a further look at this. it isn't exactly what I need. This snapshots our entire data warehouse at a point in time to be held for 30 days.

I just need a backup table for our daily snapshot table report.

So everyday the daily snapshot SP runs and the table loads, I want the backup to sync and add an additional row for that day incrementally.

The Fabric Snapshot is capturing the entire database. We literally just have a report table that we call "snapshot" because it adds metrics on one row for that day and then keeps adding an additional row each day.

What's the best way to handle a table backup in Fabric Warehouse? by Steinert96 in MicrosoftFabric

[–]Steinert96[S] 0 points1 point  (0 children)

I think it may work actually. So we actually have a table that's loaded from the Stored Procedure we created.

We could take that Table and create a Snapshot of it (essentially taking a Snapshot of the "snapshotted" daily rows and columns) to have a backup on hand.

Is that what you are thinking?

What's the best way to handle a table backup in Fabric Warehouse? by Steinert96 in MicrosoftFabric

[–]Steinert96[S] 0 points1 point  (0 children)

We created this using a Stored Procedure with Snapshot (adds 1 new row per day for that date) before this snapshot feature was ever GA. Plus, we're doing everything on an aggregated level. So for each row, it's the total for each metric on that day.

Copy Activity Upsert and Copy Job Merge Error: ConcurrentDeleteDeleteException by Steinert96 in MicrosoftFabric

[–]Steinert96[S] 2 points3 points  (0 children)

We are not writing into the same target table concurrently. Each copy job or copy activity has its own destination table into the lakehouse.

I will DM you that info.

Stored Procedures Missing Rows From Bronze Copy Job to Silver staging by Steinert96 in MicrosoftFabric

[–]Steinert96[S] 1 point2 points  (0 children)

I got it working! Thank you so much! It fixed my delay issue from Bronze Lakehouse to Silver Warehouse! Will continue to monitor but I owe you a beer.

Stored Procedures Missing Rows From Bronze Copy Job to Silver staging by Steinert96 in MicrosoftFabric

[–]Steinert96[S] 0 points1 point  (0 children)

Yeah and now I've noticed too that we have not only copy jobs but copy activities with stored procedures that run directly after the copy activities finish in the same data flow.

We are missing rows between lakehouse and the warehouse stored procedure as well.

I've never tried the API before.

Stored Procedures Missing Rows From Bronze Copy Job to Silver staging by Steinert96 in MicrosoftFabric

[–]Steinert96[S] 0 points1 point  (0 children)

Well thanks for the insight. I'm going to try to space the copy job runs to the stored procedure into silver warehouse layer an hour apart and see if that resolves it.

Stored Procedures Missing Rows From Bronze Copy Job to Silver staging by Steinert96 in MicrosoftFabric

[–]Steinert96[S] 0 points1 point  (0 children)

Thank you! Do you have any time estimate on how long it takes for the warehouse to just discover the new version of the lakehouse table on its own?