I have spent many long weekends getting stuck into Azure Databricks, plenty of time to understand the core functionality from mounting storage, streaming data, knowing the delta lake and how it fits into the bigger picture with tech like Event hubs, Azure SQL DW, Power BI etc.
So, I am going to show you how easy it is to create a cluster (that’s the end goal), you will appreciate the ease of deployment for huge amounts of infrastructure.
It has been a while since I posted an entry for TSQL Tuesday, which, for today is hosted by Kenneth . The subject being a non SQL Server tip. For a while now I have been using other technologies besides SQL Server and recently Azure Databricks and I have a handy tip for when starting this journey. It is not ground breaking but useful!
Recently I got to a stage where I leveraged Databricks to the best of my ability to join couple of CSV files together, play around some aggregations and then output it back to a different mount point ( based on Azure Storage) as a parquet file, I decided that I actually wanted to move this data into Azure SQL DB, which you may want to do one day.
All About Tech 