As I mentioned in my last post, I’ve been testing several different cloud data warehouses. Though I considered AWS Redshift and Google Big Query, I concluded they don’t have the feature set or compatibility with other tools that we need since our systems are fairly complex and relational and not at the enormous petabyte scale that Redshift and Big Query are designed for. I settled on snowflake and Azure as my final two options. My first choice was to try Azure’s offerings since I assumed that would be the most painless transition from our old SQL server databases.
My conclusion is that Microsoft is heading in the right direction but they still have a long way to go before making a truly cloud-optimized platform. They currently offer Azure SQL Database as well as Azure SQL Data Warehouse. SQL Database is basically just a cloud version of traditional SQL server. I moved several databases up there to test it. The two main things I liked over on-premise SQL Server were that you can scale the performance up and down as needed and they have introduced some Artificial Intelligence based database optimization tools which are a big help. The scaling of the service was much easier than with an on-prem installation (obviously) but it was still quite complicated to automate requiring you to build a powershell script to scale it up and down on a regular basis over night, for example. You’d think Microsoft would just create a simple auto-scaling or scheduled-scaling GUI to do this.
Continue reading “Testing Snowflake vs Azure”