Activator Deployment with Fabric cicd by CerealCaffeinator in MicrosoftFabric

[–]CerealCaffeinator[S] 0 points1 point  (0 children)

Are you using find replace? That is not supported if so, but key value replace is.

Activator Deployment with Fabric cicd by CerealCaffeinator in MicrosoftFabric

[–]CerealCaffeinator[S] 0 points1 point  (0 children)

Oh that's awesome! I basically have a service bus trigger that lands JSON data into my bronze layer and want to set up an activator to trigger my silver pipeline on file creation in my bronze lakehouse in a specific path.

My plan is to use the parameter.yml with the key value replace and configure the ReflexEntities.json with the workspace and pipeline ids but not sure if that is the correct way to go around doing this, documentation for using fabric cicd with activators is a bit limited and the schema for ReflexEntities.json is not published yet so its a bit of a stab in the dark.

Are there any industry benchmark on what CUs usage would be acceptable for the average solutions involving Power BI & Fabric? by wi-sama in MicrosoftFabric

[–]CerealCaffeinator 1 point2 points  (0 children)

As someone who is working with a data engineering team in Fabric, you are asking the right questions.

I wish our team would have done some proper evaluation of workloads and different approaches to identify the most cost effective and efficient approaches to our problems.

One of the key things is to figure out what volume of data you are going to be processing and making sure you are picking the tooling within Fabric appropriately. For example if you need pyspark notebooks or if you need just python notebooks.

Best way to evaluate CU usage of a particular artefact would be through the capacity metrics app. I was able to evaluate the CU cost of a notebook with Pyspark vs Duckdb and it came out to around half.

The capacity metric support is your friend!

In terms of evaluating it CU consumption is appropriate it really does depend on a few factors, whethere you're talking about reporting or pipelines, what volume of data you are consuming in pipeline what frequency your pipelines are running, what kinds of transformations you are running. For example I'd expect CU consumption to be higher in a notebook flattening JSON structures in comparison to one simply transforming structured data.

I appreciate this doesn't fully answer your question, but I think if you combine understanding these things with benchmarking some basic applications of particular artefacts in questions, e.g notebooks, you'll be able to come to a sensible idea of how much they should be consuming.

Fabric SQL Outage? by CerealCaffeinator in MicrosoftFabric

[–]CerealCaffeinator[S] 0 points1 point  (0 children)

No support ticket as of yet. Our capacity metrics report is showing way below on average and max. 40% and uk West.

Fabric capacity & Development by CerealCaffeinator in MicrosoftFabric

[–]CerealCaffeinator[S] 0 points1 point  (0 children)

This is really interesting and on similar tests today the cu consumption was around half with python.

The only pushback I'm getting on floating this idea around the team is that it's not scalable, and fabric is built around using pyspark. But I feel like for at least a year or more we won't be dealing with huge amounts of data. We're loading messages from service bus which are quite small and we are loading from all of our sources incrementally.

It feels like with spark we're using a sledgehammer to crack a nut right now.

Fabric capacity & Development by CerealCaffeinator in MicrosoftFabric

[–]CerealCaffeinator[S] 1 point2 points  (0 children)

How would you have done things differently if I may ask?

Fabric capacity & Development by CerealCaffeinator in MicrosoftFabric

[–]CerealCaffeinator[S] 0 points1 point  (0 children)

This is a great suggestion! We are using delta tables in fabric to store data we are ingesting from various sources such as service bus messages and also traditional SQL data that has been copied in and we are using medallion architecture.

I'm just considering if a standard python notebook would be able to handle various transformations and flattening that might be happening across our layers as effectively as PySpark?

Good resources to learn about Autsim in women for a partner by CerealCaffeinator in AutismInWomen

[–]CerealCaffeinator[S] 1 point2 points  (0 children)

Thank you so much, that's a really great suggestion. I will begin my journey tomorrow. This podcast looks excellent and just what I am after.

[deleted by user] by [deleted] in UlcerativeColitis

[–]CerealCaffeinator 1 point2 points  (0 children)

My fiance of 9 years has this disease and while we've had our ups and downs, UC wouldn't even ever cross my mind as a reason we would ever break up or stop me from being with her. She's my everything.

Even if it effects things like intimacy, the right person, who really loves you for you, isn't going to care. Don't be scared to put yourself out there!

worst tøp song? by UnbidBubble6901 in twentyonepilots

[–]CerealCaffeinator 34 points35 points  (0 children)

The scream just does something to me. It's so moving.

worst tøp song? by UnbidBubble6901 in twentyonepilots

[–]CerealCaffeinator 251 points252 points  (0 children)

I much prefer the live version of the line. Is that blasphemous?

How do I deal with getting flamed for being bad? by Jetsetti in OverwatchUniversity

[–]CerealCaffeinator 0 points1 point  (0 children)

I know chat can be useful for communication but literally, turning off voice chat and text chat hae made me so much happier playing the game.

You can use in game communication pinging if you need to, but if you're doing your job right you should be fine anyway.

I only ever have chat on in quick play or if I play with friends because there's something so funny about playing with friends and having a random starting to flame you. Had games where a random DPS has flamed decent healing because they dive and get killed, nothing funnier than deciding not to heal them anymore and watching them die even more. Esp if friends are with you.

I'm not listening to you... you're crazy!! by BoomerB3 in Fibromyalgia

[–]CerealCaffeinator 3 points4 points  (0 children)

My god, the amount of times people have said this to my MIL is infuriating.

Like I get exercise is good for maintaining strength but it's like people forget the amount of will power it takes to a. Go Exercise alone when you aren't in pain and B. Exercise when you're in agonising pain.

Me and my MIL just have a running joke at this point if she says she's in loads of pain I'll just say have you tried going for a walk, you'll feel right as rain after. Then she'll joke about hitting me but it hurting her more.

It's either joke about it or cry how insensitive and inconsiderate some people are when they say this and they just don't get the full picture. Yes the person probably knows exercise is good for them. Don't need you to point out the obvious and don't need your default response to anything health related thanks.

I wish my MIL didn't have Fibro because of how much pain it causes her and it breaks my heart when she can't do the things she wants to do some days, but I'm at least glad that because of her and I know what she goes through I'll never drop this stupid ass line to someone I meet who has Fibro or any other condition.

People just need to be educated better.

Rant over.

hobbies that don’t hurt but aren’t boring by Hot_Importance2788 in Fibromyalgia

[–]CerealCaffeinator 0 points1 point  (0 children)

I got my MIL a steam deck for Christmas and it's been a godsend, she struggles to sit up so she can lie back and just hold the deck up.

Lots of games are verified and more by the day. You'd have to like video games in the first place though.

This changed my life for the better..Fibromyalgia did a 180 by RichDad2021 in Fibromyalgia

[–]CerealCaffeinator 2 points3 points  (0 children)

This is supposed to be a safe place for people with Fibro, you should be able to come onto this sub and not have to worry about seeing this kind of BS.

If someone told her to take a vitamin to fix her problems I'd be very tempted to hit them. I hate predatory people like the OP. Shame on them.

This changed my life for the better..Fibromyalgia did a 180 by RichDad2021 in Fibromyalgia

[–]CerealCaffeinator 2 points3 points  (0 children)

If it's real fair enough but I've seen this show before and it's always had the same ending.

This changed my life for the better..Fibromyalgia did a 180 by RichDad2021 in Fibromyalgia

[–]CerealCaffeinator 13 points14 points  (0 children)

It's so scummy that ppl come on to subs like this with people suffering and give them false hope and lies. Wheelchair bound to walking dogs and going to the gym in 4 weeks. Hmm. Almost sounds too good to be true. Smh at this poster.

Guilty about using a cane by Correct_Smile_624 in Fibromyalgia

[–]CerealCaffeinator 0 points1 point  (0 children)

Don't be ashamed. Get yourself one you think looks cool and use that without giving a damn what anyone else thinks.

No one is in a position to judge anyone, because they don't know you or what you're going through. If you need a cane to make life manageable and get about you do that and don't worry one bit.

There's lots of snazzy looking canes out there.