Technically you do not have to create a cosmos DB and incur costs to test cosmos DB based applications, you could use the local emulator. This means that you do not need an Azure subscription, an actual hosted database or even an internet connection, everything is local to your machine and once ready you can deploy the solution to the cloud.
Cosmos DB falls into the “NoSQL” technology group, call it a buzz word if you like but it is very different to classic Relational Databases such as SQL Server. I never really understood why it was labeled NoSQL, if you build a document based data model via the SQL API you actually use SQL like language to query the JSON (from the collection). Moving on, it is an evolution from what used to exist called Document DB. (Cosmos DB sounds way more aggressive and punchy, right?) Recently I have been using it to store JSON documents (Document based data model) in Azure, as a fully managed service, i.e. PaaS and maybe you will use it one day.
Quite simply the objective as follows: Move data from Azure SQL Database to Azure SQL DW via Azure Data Factory v2 (ADF).
Quite a significant change has taken place within the Azure SQL Database space, more specifically the development of Azure SQL Database Serverless. Currently in preview mode this “compute” tier changes how you are billed (/second) and addresses some behaviors that many have wanted in the past. There are things to be aware of though.
Before writing about pausing (and resuming) Azure SQL Data Warehouse (DW) it makes sense to discuss the architecture of this product. At a high level it involves a control node, a MPP (Massively Parallel Processing) engine compromising of compute nodes and storage. Perfectly summarised by this image.
A quick 2 minute upload (with sound, my voice) showing you how easy it is to create an Azure SQL Database using the Azure portal and then using SSMS (SQL Server Management Studio) to connect to it.
There are a few ways to scale a SQL elastic pool. For this blog post I show you how to scale up. It can be done via the Azure portal and Azure PowerShell but not T-SQL.
I would say the PowerShell route is the easiest. Connect to your account and issue the below code. Here I am going from a 100 edtu pool to a massive 2000 edtu pool whilst tweaking the min/max setting.
The below image is a beautiful picture, now it could be worse. The red line and the green line could peak at the same time and for a very long time or the blue line could behave the same as the red line and peak at the same time as the green line. Regardless of the situation the point of this blog post is when you are hitting your eDTU (elastic database transaction unit) limits within your elastic pools, tune your queries and do not knee jerk and just scale up (straightaway that is).
There is a new way to setup Azure elastic jobs to run against a target group of databases (targeting an elastic pool). I actually found the process quite messy ( especially when trying to setup all the security access via T-SQL). Setting it up via the Azure portal is not currently an option. The first element remains the same, the need to create a “Job database”. This is the central database, think of it as the master. (Not really the master database in SQL Server but holds a lot of metadata etc) Then you need to define the group (usually the elastic pool), create credentials for access and then the job itself.
Do you want to identify the correct Service Tier and Compute Size ( was once known as performance level) for your Azure SQL Database? How would you go about it? Would you use the DTU (Database Transaction Unit) calculator? What about the new pricing model vCore? How would you translate you current on-premises workload to the cloud?