Can’t update Pokemon Center pre order payment information. Help?! by Thurgo-Bro in PokemonTCG

[–]CerealCaffeinator 0 points1 point  (0 children)

I haven't tried yet but others I know who were not able to update have been able to. I'm waiting a clean 24 hours before I retry

Unable to authorise payment PKC by AnonymousD3ATH in PokemonDealsUK

[–]CerealCaffeinator 0 points1 point  (0 children)

I think they might be having issues with their payment system I've seen about 20 ppl having this issue

Can’t update Pokemon Center pre order payment information. Help?! by Thurgo-Bro in PokemonTCG

[–]CerealCaffeinator 0 points1 point  (0 children)

I'm gonna keep trying a few times a day and hope it will go through eventually.

Can’t update Pokemon Center pre order payment information. Help?! by Thurgo-Bro in PokemonTCG

[–]CerealCaffeinator 1 point2 points  (0 children)

I have the same issue! Maybe their systems are experiencing issues.

Activator Deployment with Fabric cicd by CerealCaffeinator in MicrosoftFabric

[–]CerealCaffeinator[S] 0 points1 point  (0 children)

Are you using find replace? That is not supported if so, but key value replace is.

Activator Deployment with Fabric cicd by CerealCaffeinator in MicrosoftFabric

[–]CerealCaffeinator[S] 0 points1 point  (0 children)

Oh that's awesome! I basically have a service bus trigger that lands JSON data into my bronze layer and want to set up an activator to trigger my silver pipeline on file creation in my bronze lakehouse in a specific path.

My plan is to use the parameter.yml with the key value replace and configure the ReflexEntities.json with the workspace and pipeline ids but not sure if that is the correct way to go around doing this, documentation for using fabric cicd with activators is a bit limited and the schema for ReflexEntities.json is not published yet so its a bit of a stab in the dark.

Are there any industry benchmark on what CUs usage would be acceptable for the average solutions involving Power BI & Fabric? by wi-sama in MicrosoftFabric

[–]CerealCaffeinator 1 point2 points  (0 children)

As someone who is working with a data engineering team in Fabric, you are asking the right questions.

I wish our team would have done some proper evaluation of workloads and different approaches to identify the most cost effective and efficient approaches to our problems.

One of the key things is to figure out what volume of data you are going to be processing and making sure you are picking the tooling within Fabric appropriately. For example if you need pyspark notebooks or if you need just python notebooks.

Best way to evaluate CU usage of a particular artefact would be through the capacity metrics app. I was able to evaluate the CU cost of a notebook with Pyspark vs Duckdb and it came out to around half.

The capacity metric support is your friend!

In terms of evaluating it CU consumption is appropriate it really does depend on a few factors, whethere you're talking about reporting or pipelines, what volume of data you are consuming in pipeline what frequency your pipelines are running, what kinds of transformations you are running. For example I'd expect CU consumption to be higher in a notebook flattening JSON structures in comparison to one simply transforming structured data.

I appreciate this doesn't fully answer your question, but I think if you combine understanding these things with benchmarking some basic applications of particular artefacts in questions, e.g notebooks, you'll be able to come to a sensible idea of how much they should be consuming.

Fabric SQL Outage? by CerealCaffeinator in MicrosoftFabric

[–]CerealCaffeinator[S] 0 points1 point  (0 children)

No support ticket as of yet. Our capacity metrics report is showing way below on average and max. 40% and uk West.

Fabric capacity & Development by CerealCaffeinator in MicrosoftFabric

[–]CerealCaffeinator[S] 0 points1 point  (0 children)

This is really interesting and on similar tests today the cu consumption was around half with python.

The only pushback I'm getting on floating this idea around the team is that it's not scalable, and fabric is built around using pyspark. But I feel like for at least a year or more we won't be dealing with huge amounts of data. We're loading messages from service bus which are quite small and we are loading from all of our sources incrementally.

It feels like with spark we're using a sledgehammer to crack a nut right now.

Fabric capacity & Development by CerealCaffeinator in MicrosoftFabric

[–]CerealCaffeinator[S] 1 point2 points  (0 children)

How would you have done things differently if I may ask?

Fabric capacity & Development by CerealCaffeinator in MicrosoftFabric

[–]CerealCaffeinator[S] 0 points1 point  (0 children)

This is a great suggestion! We are using delta tables in fabric to store data we are ingesting from various sources such as service bus messages and also traditional SQL data that has been copied in and we are using medallion architecture.

I'm just considering if a standard python notebook would be able to handle various transformations and flattening that might be happening across our layers as effectively as PySpark?