SQL Azure Federations – true scalability is almost here

December 7th, 2011 Add Your Comments

The Windows Azure platform has been with us for a few years now, providing the means to develop and deploy highly available and scalable applications. That being said, the delivered scalability was only half of what real life applications needed. Let me explain why.

At the highest level, the server side of each application is made out of two main components: the application layer and the data layer. The application layer is responsible for implementing any server side logic, which might be UI or business related, and the data layer is responsible for storing the data. As an example, in an ASP .Net application, the application layer would be the ASP .Net application itself and the data layer would be the database server used by the application.

Single app single db

Inside Windows Azure, the application layer is implemented with the help of web and worker roles. Since any application deployed to these roles is constrained to be stateless (no data is persisted locally between two external requests), it can be easily scaled out. That means it can be deployed to multiple machines, each of them being able to interchangeably serve client requests and in this way split the load.

Multiple apps single db

Even since its beginnings Windows Azure made it very easy to add or remove virtual instances to web and worker roles and thus made horizontal scaling at the application layer very easy. However, as mentioned at the top of the post, this only solves half of the problem and that is because in the end, all instances from the web or worker role would get its data from the same database.

One classic and up to a point very efficient way of increasing the power of a database is vertical scalability, which means that when the load requires it the database should be moved to a more powerful machine. The hardware nowadays is very powerful and mature database engines like the SQL Server are able to squeeze every drop of performance out of it.

Multiple apps big db

There are a few fundamental problems with vertical scaling though. The first one is that it can only be done up to a limit and that is the performance limit of the hardware available on the market. Then from a certain point upward, the price of the hardware rises exponentially and it doesn’t become feasible anymore. Also, this kind of scalability involves big upfront budgets being spent on hardware.

The final problem is that scaling up is only an option when actually owning the servers. With cloud providers you rarely have the option to run databases on dedicated hardware and even then your choices are limited.

In SQL Azure, the relational database engine provided by the Azure platform, administrators have no way to set the physical resources that a server is using, so vertical scalability is not an option. Furthermore, behind the scenes, each SQL Azure instance is deployed on a shared server. Microsoft are claiming that those servers are actually very well optimized for running databases, but there is no getting away from the fact that they are shared and you will probably get a better performance out of an Sql Server instance deployed on a relatively cheap dedicated server.

This meant that for many applications, the scalability limit of Azure deployed applications wasn’t very high. But all this should change with the arrival of SQL Azure Federations which are scheduled to be released by the end of 2011. You can read more details on the SQL Azure Federations here, but in a nutshell, they provide the mechanism the spread the contents of a database through multiple instances. Most importantly, adding or removing instances from a database does not require any downtime!

Multiple apps multiple DBs

Up until now, SQL Azure really presented only one major advantage over on premises databases and that was that it ensured durability and availability of the data by always keeping at least three copies of each database. Now, with the addition of Federations it has also become scalable. Still, there are some features still missing (like the already planned full text indexing), but never the less a lot of real life applications can now benefit by being deployed to Azure.

Update Starting with December 10th 2011, Microsoft has publicly released the SQL Azure Federations feature.

PS: Besides SQL Azure, Microsoft is also offering a NoSql solution, called Azure Table Storage, which right from the start was designed with scalability in mind. The reason I haven’t mentioned it in this article is because I find its limitations to be too important to use it in anything else than a few particular scenarios.

Comments are closed.