Quite simply the objective as follows: Move data from Azure SQL Database to Azure SQL DW via Azure Data Factory v2 (ADF).
Advertisements
Using Azure Data Factory to Migrate Data
Reply
Category Archives: Azure
Azure SQL Database Serverless
Quite a significant change has taken place within the Azure SQL Database space, more specifically the development of Azure SQL Database Serverless. Currently in preview mode this “compute” tier changes how you are billed (/second) and addresses some behaviors that many have wanted in the past. There are things to be aware of though.
Pausing / Resuming Azure SQL DW
Before writing about pausing (and resuming) Azure SQL Data Warehouse (DW) it makes sense to discuss the architecture of this product. At a high level it involves a control node, a MPP (Massively Parallel Processing) engine compromising of compute nodes and storage. Perfectly summarised by this image.
Video – Creating Azure SQL Database
A quick 2 minute upload (with sound, my voice) showing you how easy it is to create an Azure SQL Database using the Azure portal and then using SSMS (SQL Server Management Studio) to connect to it.
Scaling SQL Elastic Pools
There are a few ways to scale a SQL elastic pool. For this blog post I show you how to scale up. It can be done via the Azure portal and Azure PowerShell but not T-SQL.
I would say the PowerShell route is the easiest. Connect to your account and issue the below code. Here I am going from a 100 edtu pool to a massive 2000 edtu pool whilst tweaking the min/max setting.
Tuning Azure SQL Elastic Pools
The below image is a beautiful picture, now it could be worse. The red line and the green line could peak at the same time and for a very long time or the blue line could behave the same as the red line and peak at the same time as the green line. Regardless of the situation the point of this blog post is when you are hitting your eDTU (elastic database transaction unit) limits within your elastic pools, tune your queries and do not knee jerk and just scale up (straightaway that is).
New Improved Azure Elastic Jobs
There is a new way to setup Azure elastic jobs to run against a target group of databases (targeting an elastic pool). I actually found the process quite messy ( especially when trying to setup all the security access via T-SQL). Setting it up via the Azure portal is not currently an option. The first element remains the same, the need to create a “Job database”. This is the central database, think of it as the master. (Not really the master database in SQL Server but holds a lot of metadata etc) Then you need to define the group (usually the elastic pool), create credentials for access and then the job itself.
Sizing your Azure SQL Database
Do you want to identify the correct Service Tier and Compute Size ( was once known as performance level) for your Azure SQL Database? How would you go about it? Would you use the DTU (Database Transaction Unit) calculator? What about the new pricing model vCore? How would you translate you current on-premises workload to the cloud?
Azure SQL Data Warehouse Query Store
I personally think that query store has been a fantastic feature. I find myself using it for query performance troubleshooting (plan regressions is a big one). This has always been available since SQL Server 2016 and even Azure SQL Database but now Microsoft have made it available for Azure SQL Data Warehouse (DW).
If you connect to the Azure SQL DW via SQL Server Management Studio (SSMS) there is no Query Store Node (for the version, I was checking 17.9.1). I was even wondering if it was on by default?
Azure SQL Database and Trace Flags
You cannot enable trace flags (globally or by session) within Azure SQL Database but did you know that some global trace flags are enabled by default?
What is on?
