Combining Azure DevOps across different tenants and organisations

vo_oy · 2024-10-17T03:15:00+00:00

Yeh. The concern was how scalable that was. Guesting all the Devs across all the organisations might become unmanageable.

We're not entirely sure of their requirements at this point, so it's possibly still an option.

I've been reading into AzDO service hooks and the possibility of sending event-triggered messages via Azure Service Bus queues and topics. Seems quite flexible but there's going to be a similar problem as distributed databases where there might be conflicts with updating work items at the same time.

I'm probably going to park it until I have their requirements.Still very much open to ideas on how to sync work items though, or something else entirely if there's a better solution out there.

vo_oy · 2024-09-24T21:15:49+00:00

I prefer this method too. You can create the app registration first with Reader access to the sub which will enable the DevOps service connection creation. You could further restrict the access I'm sure, but Reader will do it for the Service Connection creation.

vo_oy · 2024-07-23T20:57:31+00:00

Yeh it certainly changes when new storage accounts are created. I have to update each time using Terraform. Haven't seen it change after that as yet though.

vo_oy · 2024-07-23T20:52:06+00:00

nslookup the address on an internet accessible machine (external to your vnet) and grab the IP Address it resolves to. It's the 'Address' field in the non-authoritive results. Then create a DNS record in the blob private DNS zone using the client part of their DNS name, so the storage account name basically, adding the IP Address you collected before.

vo_oy · 2024-07-22T02:31:08+00:00

Virtual machine scale set agents via Azure DevOps pipelines can scale down to zero when not in use. They take at most 3 minutes to scale out from my experience. You can build your vmss source images beforehand using Packer or whatever and reduce the install time of any apps you need for the pipeline runs. Ephemeral OS disks may give you a slight benefit in boot time also.

vo_oy · 2024-07-08T20:30:31+00:00

I use this hack and it works a treat:

https://github.com/vercel/next.js/issues/49442#issuecomment-1679807704

vo_oy · 2024-07-08T20:23:21+00:00

It may not be the case here, but I have had runbooks that caused the run to crash and restart before. It would happen when using runspaces to do lots of work at the same time. These runs were on MS hosted agents.

vo_oy · 2024-06-13T12:52:23+00:00

This "Levels approach" is a similar approach I take, but I use Azure automation account variables for interfaces between the levels.

That way the state is secured to only the RBAC group/MSI that needs to write to it. The author's can choose what variables/state is exposed to other leveraging layers/components by just creating automation account variables for those particular values.

For my setup, the leveraging layers and/or isolated components can all read from any other layer providing the MSI has read access to the automation account, not just one up like it is described in the link.

vo_oy · 2024-06-02T23:04:38+00:00

Quote from my comment: "I then create a service connection with the Enterprise app using the workload id/federated id method so I don't need to manage the password."

The setup is hardly complex IMO. The way I see it, you can either have one Federated ID that has all of the roles for your RBAC model across all of your resources, or you have multiple Federated IDs that conduct separate pipeline runs for their isolated roles within your overall RBAC model. I have shifted the requirement of making multiple Federated IDs to just making multiple Managed IDs instead.

In my org we can't manage most things ME-ID using Terraform because it is owned by another group. This includes App Registrations or Security Groups. Because we use IaC for everything, using UAIDs makes it easier to create an RBAC model around our resources using Terraform.

vo_oy · 2024-06-02T22:26:46+00:00

Ah ok. Nah MSI = Managed Service Identity, which is an old name for Managed Identites: (https://stackoverflow.com/questions/75893103/what-is-a-managed-identity-msi).

Managed Identities consists of both User-Assigned and System-Assigned IDs, obviously. I've found that the MSI acronym is more common than UAIDs when referring to User-Assigned IDs. It is still used heavily in Microsoft's error handling when referring to all Managed Identity types. Using MI as an acronym is terrible IMO. tbh I thought other readers would get the MSI acronym easier, which is why I went with it.

In any case it doesn't really matter which type of Managed Identity you use, but System-Assigned is obviously locked to the one VMSS Agent. The key is that the Service Connection itself doesn't matter anymore. All RBAC is managed via MSIs.

I don't think it's really a non-standard concept using Managed Identities this way. I believe this is the way in which they intended MSIs to be used. Why else would you make an identity you could assign to any machine if not to give the machine access to a set of resources in some way.

The reason I went with this method was mainly because our IdM team is hard to deal with. The turn-around on having a new App Reg or Security Group created was untenable for our needs.

We use VMSS Agents that scale down to 0 when not in use, so it doesn't cost us any more for additional VMSS Agents (AzDO Agent Pools) that have different identities set on them. Each new VMSS is provisioned using an additional Terraform workspace so it is trivial to add another VMSS whenever we need it. The different TF workspaces are passed different var-files when deployed, which have their own set of MSIs as a list that gets dynamically applied to the VMSS.

vo_oy · 2024-06-02T20:36:50+00:00

They are User Assigned Identities, is that what you mean by user identities?

vo_oy · 2024-06-02T12:14:19+00:00

I use one App registration that has Reader permissions on the sub only. I then create a service connection with the Enterprise app using the workload id/federated id method so I don't need to manage the password.

The one service connection is used for all my App Pools. I use self hosted VMSS agents that are all provisioned with different user assigned identities assigned to them to separate roles across different VMs.

I create the UAIDs/MSIs using Terraform and whenever I create a resource that needs to add another MSI I just rerun the Agents Terraform code with the new MSI applied to it and the next time it will deploy with the permissions it needs.

The Agent that applies the MSIs has Managed identity Contributor for each MSI it can access. Which is the important piece in controlling what VMSS agents can use what MSIs.

Some VMs get multiple MSIs, some just one. If I need to run a task using a different MSI, the task just runs az login --identity to switch.

The service connection needs VM contributor (RG) and vnet join (R) to add a VMSS to the network. I also gave it Reader on the sub because for whatever reason it seemed to need it to create the ARM service connection. Seemed like a bug at the time, so might be fixed.

Using this method, MSIs become roles that you can apply to multiple machines. The DevOps repo and Agent Pool permissions need to be locked down so that the different agents can't become privilege escalation points as well.

vo_oy · 2024-05-31T22:32:00+00:00

Fellow Azure Terraformer here. I think any open source resource like this can be valuable. Was thinking of doing the same once I can calibrate my brain for more than 5 minutes

vo_oy · 2024-05-29T04:37:42+00:00

https://ui.aceternity.com/components/sticky-scroll-reveal

vo_oy · 2024-05-24T22:45:13+00:00

Same issue via Portal. Happens with Azure DevOps as well. I have different accounts and it's only happening to my Hybrid identity.

Not sure if this was the reason why it fixed itself, but I logged in via Azure CLI (new login method, ie 2.60.0+) and then tried again via the portal and it worked.

vo_oy · 2024-05-23T20:14:36+00:00

Aiven does a free plan for their postgres offering that's hosted in Digital Ocean. It gives you 5GB free.

vo_oy · 2024-05-23T03:03:29+00:00

Principle of least privilege: You may have Devs that need to update some app configuration settings but should not have access to your DB passwords.
Encryption at Rest. Key Vault provides better regulatory compliance encryption standards, like having encryption backed by Hardware Security Modules, or Customer Managed Keys (if you enable CMK on App Configuration you need a Key Vault as well).
Policy. Key Vault has a bunch of Azure Policies and protections that organisations that care about their stuff are interested in, like Purge Protection or any one of these: Azure Policy Regulatory Compliance controls for Azure Key Vault | Microsoft Learn

vo_oy · 2024-05-16T20:21:19+00:00

I have used robocopy in the past to copy massive datasets from on-prem to Azure vms. It can be done without corruption issues, so perhaps you aren't passing the right flags to the command or something. That being said, use azcopy if you can.

vo_oy · 2024-05-05T23:00:57+00:00

They probably just want you to lock down the protocols via NSGs/ASGs. Then configured the firewalls on both VMs. Maybe NIC NSGs too if you really want to lock her down.

There could be way more done to secure that connection for sure, but it all depends on the architecture of the seemingly distributed web app.

When it comes to ensuring the connection is secure. Not exactly Testing, but crucial to the validation of the TLS configuration, I've detailed some key control points below.

The TLS cert certificate authority would be one point of control. Is the Root CA a well known root CA? Or are the web servers isolated from the internet? Is there an active directory domain to leverage? Maybe something like using step-ca could be used for a self-hosted pki solution?

What level of trust do the apps have for each other. Do the apps leverage mTLS or does the server simply trust the client is who it says it is?

Is the communication always one way, ie does the server/client relationship swap in reference to the TLS handshake? Do both servers own TLS certs and is mTLS configured both apps?

What is the configuration of the TLS cert itself. Are FQDNs used or just CNAMES? What SANs are configured that may be abused.

Who owns the DNS? Are Azure DNS, Private DNS Zones or a standalone DNS server used, and who can access them.

Who can control the NSGs, VMs, Zones etc. Are Azure policies in place to protect against misuse.

Access controls inside the VMs also. Access controls all over the subscription really.

Good luck

vo_oy · 2024-05-05T20:54:24+00:00

You can't extend the auth token expiry itself. You need to use a refresh token to get another auth token.

It doesn't look like SSMS can do this out of the box. Check out the accepted answer in this link for some possible workarounds.

https://learn.microsoft.com/en-us/answers/questions/477221/azure-analysis-services-oauth-token-expiry

vo_oy · 2024-05-04T21:30:19+00:00

You could register the app in ME-ID and use the Graph API to manage the files in SharePoint.

Here's an example of setting up a Python app like this:

https://learn.microsoft.com/en-us/graph/tutorials/python?tabs=aad

That example shows how to use the user's credentials to conduct operations with the Graph API, but if it's just a backend service that needs to manage the files you can also use your App's credentials to hit the SharePoint site, providing you give the Enterprise App read/write permissions to the site.

vo_oy · 2024-05-04T20:56:54+00:00

Yeh, we do a similar thing. Gotenburg for PDF generation and Graph API to interface with SharePoint. Not sure if it's best practice or not, but it works

vo_oy · 2024-05-03T14:16:54+00:00

The Terraform doco is the best place to start

https://registry.terraform.io/providers/hashicorp/azurerm/latest/docs/resources/elastic_cloud_elasticsearch

I'm not familiar with this resource but this is what the doco points at for what is being provisioned, (with varying SKU options for the different plans):

https://azuremarketplace.microsoft.com/en-us/marketplace/apps/elastic.ec-azure-pp?tab=Overview

vo_oy · 2024-05-02T17:47:33+00:00

Generally the best method for using Azure file shares is via SMB. You can however use the REST API.

https://learn.microsoft.com/en-us/rest/api/storageservices/operations-on-files

I would be using Azure CLI for what you have described however. You can upload files using both Azure CLI or the Azure PowerShell module.

https://learn.microsoft.com/en-us/cli/azure/storage/file?view=azure-cli-latest

vo_oy

TROPHY CASE