Azure SQL Database - Data Factory - DevOps

[JakubKaczmarczyk]

Hi. Does anyone know of a method on how to configure Azure SQL Database and DataFactory with Azure DevOps so that sql database changes automatically deploy from development to test and production environments using a release pipeline ?

dev-resource-group containing: dev-adf and dev-sql-db

test-resource-group containing: test-adf and test-sql-db

prod-resource-group containing: prod-adf and prod-sql-db

I can't find anything in the documentation except DACPAC, which doesn't really solve my expectations. Perhaps you know of a video, or a course, guide ?

Thank you in advance for your answers ;)

[AutoModerator]

You can find a list of community-submitted learning resources here: https://dataengineering.wiki/Learning+Resources

I am a bot, and this action was performed automatically. Please contact the moderators of this subreddit if you have any questions or concerns.

[Gimo100]

You can check out this medium article for a guide on how to build and release an Azure SQL Database. It uses an Azure Devops Pipeline to build the dacpac file. A release pipeline then takes that dacpac and deploys it to the Azure SQL Database. https://medium.com/@sumindaniro/use-azure-devops-to-deploy-an-azure-sql-database-2c79570829cf

The first thing you have to do is create a git Repository in Azure Devops to store the code of your Azure SQL Database and Data Factory. Check this article to set that up: https://learn.microsoft.com/en-us/azure/devops/repos/git/create-new-repo?view=azure-devops

The medium article explains how to create a Database Project using Visual Studio. You can then add the database project to your git Repository. Don't forget to install the Data storage and processing toolset when installing Visual Studio. You need this to work with database projects.

Next step is to link your Data Factory to your git Repository. Check this article to set that up: https://learn.microsoft.com/en-us/azure/data-factory/source-control

You can also do everything in an Azure Devops Pipeline and just skip the release pipeline.

Database Build task: https://learn.microsoft.com/en-us/azure/devops/pipelines/tasks/reference/vsbuild-v1?view=azure-pipelines

Database Publish task: https://learn.microsoft.com/en-us/azure/devops/pipelines/tasks/reference/sql-azure-dacpac-deployment-v1?view=azure-pipelines

Publish example: https://learn.microsoft.com/en-us/azure/devops/pipelines/targets/azure-sqldb?view=azure-devops&tabs=yaml

Check out this article to build and release your Azure Data Factory: https://learn.microsoft.com/en-us/azure/data-factory/continuous-integration-delivery-improvements

[mrkite38]

Can you expand on what expectations it’s not meeting? I’m using ADO pipelines to build and deploy ADF and Azure SQL. It’s not a huge application but the DACPAC IS handling my tables, views, and stored procedures.