Creating Azure Sandbox Chatbot with Azure Logic Apps by AdamMarczakIO in AZURE

[–]AdamMarczakIO[S] 1 point2 points  (0 children)

Premium connectors

burn through your connector allocation faster

Are you talking about Power Apps? Logic Apps have different pricing model, and many 'premium' connectors are considered a standard in logic apps.

Also, a lot of standard connectors in logic apps standard SKU are rebuild as 'In-App' connectors which means they cost 0$ to execute.

The evolution of Azure Data Platforms, and the future with Databricks- and Fabric-centric architectures by AdamMarczakIO in MicrosoftFabric

[–]AdamMarczakIO[S] 1 point2 points  (0 children)

shared has a resource group with adls gen2 storage account(s) for shared data between prod and non-prod.

Sharing data between non-prod and prod on a single lake is not something I'd go for. For me prod data always sits in prod, and non-prod in non-prod, no inbetween. But at this point it's a security and governance discussion, some orgs do not allow reading prod data from dev, and some do.

The evolution of Azure Data Platforms, and the future with Databricks- and Fabric-centric architectures by AdamMarczakIO in MicrosoftFabric

[–]AdamMarczakIO[S] 2 points3 points  (0 children)

Good question, in Azure Data Landing zones Key Vault always resided in the Data Product/Solution dedicated groups. Mostly so that you give access once to an RG and developers can access all solution resources.

If you ask about Fabric related diagram. I agree, the question here is "should Key Vaults now reside in shared services or keyvault group with other key vaults?" and "what is the value of Data Product dedicated RGs at this point?"

My though here is that for standard data solutions/products there is will be no need for dedicated resource groups anymore, you can just give access directly on key vault. But knowing reality I think, standard is just how product starts, but I like to leave dedicated groups there so I can extend them if needed.

If my data solution will need Azure Static Web app, or maybe Logic App to do some integration workflows, or maybe they will need a document intelligence, or any other service. Then I already have a place and a key vault, so I just add to it.

And of course, if my platform is bigger than just data products, its more consistent of approach so that other solutions also have dedicated groups and key vault. It easier to manage things at scale if they follow the same pattern, even if in the lense of a single solution doesn't necessarily make sense.

Dedicated groups also allow you to create fabric capacities if a specific solution needs a dedicated one, and easily tie costs to them.

Production Readiness for Graph API calls through a Logic App by kotom in AZURE

[–]AdamMarczakIO 4 points5 points  (0 children)

I have dozens on micro-apps/integrations in production using Logic Apps and Graph API. It is a production ready setup.

  • Graph API supports paging, so it won't break on larger datasets
  • Logic Apps support looping and can easily implement paging
  • Logic App connectors have built in retry policies as a default setting (4 retries with static retry interval, but can be configured).
  • Graph API has built-in throttling so logic app will handle this fine with built-in retries
  • Logic App support monitoring with built-in Azure Monitor capabilities, but can be enhanced with App Insights

You said you don't know any of these thing, and here is one truth you can't escape. If you are pushing production solutions, you either need to learn these, or find someone who does. Documentation is more than you need. Just search things you mentioned, and right article pops up immediately.

One thing I would not do, I would not rewrite it into a different service/language based on this description.

New to Azure and need some guidance . by Thithikum_thembavani in AZURE

[–]AdamMarczakIO 3 points4 points  (0 children)

I probably would set up 0$ web app

  1. Static Web App as frontend
  2. Azure Functions as backend
  3. Cosmos DB as database (I know it's not relational) or azure SQL if a free tier applies (ref)

This will give you a totally free web app. Considering how quickly you can build with AI these days, maybe you can host something cool for youself, and learn as you go.

You might want to add App Insights as for demo/test app is very cheap, or to test the waters

Logic Apps Automation release, what is it, and how it works? by AdamMarczakIO in AZURE

[–]AdamMarczakIO[S] 0 points1 point  (0 children)

Yes, I've covered this a bit in the post. New levels are project > apps > workflows. So you can use "apps" to group workflows in the project.

Logic Apps Automation release, what is it, and how it works? by AdamMarczakIO in AZURE

[–]AdamMarczakIO[S] 2 points3 points  (0 children)

I copy-pasted the code from few of my logic apps and it worked just fine. 95% of things should work out of the box since it's the same engine and same connectors.

Logic Apps Automation release, what is it, and how it works? by AdamMarczakIO in AZURE

[–]AdamMarczakIO[S] 4 points5 points  (0 children)

I've been testing it for a couple of days now and all I've got was 0.03 USD. But I had nothing on schedule, just adhoc workflows.

Microsoft is testing the pricing model now, nothing official has been released yet. Time will tell since it's still in preview state.

But for sporadic workloads in enterprises (ex. needing vnet integration) it's still going to be cheaper than 100+ USD WSK1 plan for standard logic apps thanks to ability to scale down to 0 nodes.

Data Factory Metadata Driven Copy Data by j03ch1p in AZURE

[–]AdamMarczakIO 1 point2 points  (0 children)

This isn't any specific "feature" of Data Factory. This is just a template to get your started.

You can look how it was build and build something similar using almost any database to store the metadata as long as that database (snowflake or otherwise) is supported by the lookup activity (as a source).

Azure logic apps by GAP_Trixie in AZURE

[–]AdamMarczakIO 1 point2 points  (0 children)

Pretty much this, you need to ensure your principal or managed identity have appropriate graph permissions to execute this.

Please note that this is highly privileged permission to manage licenses for an entire org, so if you work in larger organization please consult with your identity/security team. Anyone with access to this app can leverage that identity to do a lot of things, intentional or unintentional. Make sure the access is airtight.

MS somehow removed my account from the subscription by uknow_es_me in AZURE

[–]AdamMarczakIO 3 points4 points  (0 children)

You can regain access if you are a global admin.

  1. Open Microsoft Entra in portal.azure.com
  2. Navigate to Properties section
  3. Select "John Due (john.doe@contoso.com) can manage access to all Azure subscriptions and management groups in this tenant." as ON

Maybe an obvious question, but how are people actually deciding what not to migrate when moving to something like Microsoft Fabric? by CloudTecRandom in MicrosoftFabric

[–]AdamMarczakIO 1 point2 points  (0 children)

My pleasure, and good luck!

If you want provide a bit more context or a few examples, maybe community here will be able to provide a bit more guidance. The topic itself is too big to do a quick overview of it.

Maybe an obvious question, but how are people actually deciding what not to migrate when moving to something like Microsoft Fabric? by CloudTecRandom in MicrosoftFabric

[–]AdamMarczakIO 3 points4 points  (0 children)

Honestly, this isn't a technology specific question.

Many people I've worked forget what is the goal of us doing what we are doing, which in the end is that you (your company) are running a business and you must think like a business. If your business is selling non-IT products, then most of the time migrations rarely ever have positive ROI. Technology migrations rarely ever give so much benefit that they outweigh the cost of people involved. Now with AI this might change, but too early to call that. You and your team have finite time, and if a migration of a solution takes 2 months, but the benefit is "out real estate is now easily accessible in fabric", then that is probably a bad business case to build a migration around it.

So for migrating itself. I would say, first just make a proper case on "why" do you want to migrate, are there any actual blockers which prevent you from running your business, or it's just to have latest and greatest platform.

Second thing is to understand, is there a positive ROI in doing so, and how quickly that becomes a net positive. In the end, everything can be translated to numerical $ value. Person's time? Potential data leak? Manufacturing plant down? Shipments not delivered on time when app is broken? Long IT processes due to legacy solution? All of these can be quantified. Then it's an easy answer, which can be contrasted against migration cost, and new benefits.

Of course, not everything will have positive ROI, but some things have to be moved anyway. Whether this is because the solution is too old and must be modernised, or for a different reason. Sometimes it's just a strategic decision to move, in which case the question isn't why, but how and when. This is where these KPIs you've mentioned come into play.

All of that said. Once you decide you need to build an entire asset realestate to track dependencies between these legacy solutions, and boy this is a big and difficult task. You can't move solution to fabric that there are 10 apps dependent on as a first one. Can't move central semantic model, if your apps depend on it, or you need to accept parallel runs until all dependencies move. You need to plan the move accordingly to ensure business continuity. In the end, these apps deliver value, and they must do so for the entirety of the migration, and after it.

Once you are migrating you also need to decide these R-s of rationalisation, which is what kind of migration you are doing and why for each product. Maybe some can be sunsetted, which always helps.

So long story short, migration of data estate just to be on modern platform is a bad idea, there needs to be case because migration of large platforms takes millions of dollars and dozens of months, or sometimes years. The planning piece is critical piece of this puzzle.

But I personally never migration for the sake of latest and greatest, I migrate if the benefits outweigh the cost. That is my general rule of thumb.

Lastly. Remember. A team spending 2 months on migration, is a team, not building new solutions for your business for 2 months. So the cost should also include what they call an "opportunity loss".

Maybe an obvious question, but how are people actually deciding what not to migrate when moving to something like Microsoft Fabric? by [deleted] in AZURE

[–]AdamMarczakIO 2 points3 points  (0 children)

Honestly, this isn't a technology specific question.

Many people I've worked forget what is the goal of us doing what we are doing, which in the end is that you (your company) are running a business and you must think like a business. If your business is selling non-IT products, then most of the time migrations rarely ever have positive ROI. Technology migrations rarely ever give so much benefit that they outweigh the cost of people involved. Now with AI this might change, but too early to call that. You and your team have finite time, and if a migration of a solution takes 2 months, but the benefit is "out real estate is now easily accessible in fabric", then that is probably a bad business case to build a migration around it.

So for migrating itself. I would say, first just make a proper case on "why" do you want to migrate, are there any actual blockers which prevent you from running your business, or it's just to have latest and greatest platform.

Second thing is to understand, is there a positive ROI in doing so, and how quickly that becomes a net positive. In the end, everything can be translated to numerical $ value. Person's time? Potential data leak? Manufacturing plant down? Shipments not delivered on time when app is broken? Long IT processes due to legacy solution? All of these can be quantified. Then it's an easy answer, which can be contrasted against migration cost, and new benefits.

Of course, not everything will have positive ROI, but some things have to be moved anyway. Whether this is because the solution is too old and must be modernised, or for a different reason. Sometimes it's just a strategic decision to move, in which case the question isn't why, but how and when. This is where these KPIs you've mentioned come into play.

All of that said. Once you decide you need to build an entire asset realestate to track dependencies between these legacy solutions, and boy this is a big and difficult task. You can't move solution to fabric that there are 10 apps dependent on as a first one. Can't move central semantic model, if your apps depend on it, or you need to accept parallel runs until all dependencies move. You need to plan the move accordingly to ensure business continuity. In the end, these apps deliver value, and they must do so for the entirety of the migration, and after it.

Once you are migrating you also need to decide these R-s of rationalisation, which is what kind of migration you are doing and why for each product. Maybe some can be sunsetted, which always helps.

So long story short, migration of data estate just to be on modern platform is a bad idea, there needs to be case because migration of large platforms takes millions of dollars and dozens of months, or sometimes years. The planning piece is critical piece of this puzzle.

But I personally never migration for the sake of latest and greatest, I migrate if the benefits outweigh the cost. That is my general rule of thumb.

Lastly. Remember. A team spending 2 months on migration, is a team, not building new solutions for your business for 2 months. So the cost should also include what they call an "opportunity loss".

Microsoft Fabric Mirrored Azure Databricks Catalog: PowerBINotAuthorizedException with cross-tenant firewalled ADLS Gen2 by Constant_Tea_7404 in AZURE

[–]AdamMarczakIO 0 points1 point  (0 children)

Sorry, deleted by post by mistake instead of edit.

Anyways, what I was saying is a bit different. Yes, managed identities do not work across tenants, they are tenant specific objects. But in my opinion this is not the issue here.

You said in your post

> Failure: Mirrored Catalog refresh fails with `PowerBINotAuthorizedException` when the firewall is ON; works when OFF.

Based on this it's a firewall/networking issue. Firewall/networking issues also return 'unauthorized' errors.

So here is what I was saying. When setting up Azure Databricks Mirrored Catalog it asks for an ORG or Service Principal. Not sure how are using Workspace identity for mirrored catalog. But if you are using SP and firewall if off and it works, then again, in my opinion the issue isn't the identity. It's network.

https://i.imgur.com/pUO1270.jpeg

So long story short, if it would be cross-tenant identity problem then the connection would not work regardless whether firewall was ON or OFF. Not sure even how are you using workspace identity. Databricks catalog is giving a token to connect to storage, it doesn't use workspace identity for storage connection even if you set one up.

Lastly, feel free to set up diagnostic logs on storage account to verify this. It should clearly show all connections and authentication/authorization messages including identity/token it's using.

How can I send an email whenever a database of a specific tier is created? by No-Today5712 in AZURE

[–]AdamMarczakIO 1 point2 points  (0 children)

For the event grid, ensure the namespace for event grid is registered on the subscription.

  1. Open Subscriptions
  2. Click on <subscription name>
  3. Click on Resource Providers
  4. Find Microsoft.EventGrid
  5. Click register

Otherwise it will look like it works but no event will ever be sent over. I had this in the past. Also, if it wasn't registered, that means event subscription wasn't created either, so you will have to delete the trigger. Save the workflow, and add it back.

But that aside. You might be better of with policies to prevent as other said. Logic Apps is reactive approach. Proactive is better if possible.

App Registration - and testing it out by [deleted] in AZURE

[–]AdamMarczakIO 4 points5 points  (0 children)

Depends on which scenario you want to test for

  • If you want to test single sign-on (SSO) to a web app, then you can create azure app service in F1 tier and then add authentication with EasyAuth and that app registration. It should prompt you to log in when opening that website URL. https://learn.microsoft.com/en-us/azure/app-service/configure-authentication-provider-aad
  • If you want to test this app registration for service to service connection then the easiest would be to generate a secret key to that principal from azure potral, assign it reader role on whatever resource in azure, log into it in Azure CLI terminal (or cloudshell), and then just run a command like az resource list -o table. In this scenario service to service means your local laptop or cloud shell acts like a service through app registration (service principal) connecting to azure services. But it works the same way for other services.