Its all about data – No Sql

Its been on my TODO list for a while now to take a proper look at the the NoSql offerings in the market. Building highly scalable architectures mean you need to think carefully about your data tier, caching, transaction management and i/o in general. Scaling systems like BizTalk meant you needed to have a very performant disk subsystem due to the high level of DB i/o. As I’ve mentioned before, this level of durability isn’t always required when building distributed systems, resiliently storing transactions in memory for example (i.e. on at least 2 nodes) might be sufficient for many applications, journaling approaches are also a good alternative as in LMAX.

I’ve recently started to look at mongoDb, under the hood it uses memory mapped files, which if you throw enough RAM at mongo it will hold all of the data in memory making it very performant. There are various options around sharding and replication of data across nodes, the trick then is to ensure writes are either replicated across nodes or the journal is flushed on write to ensure data is not lost. I’ve worked on a  few architectures over the last few years where we’re used a distributed cache in front of a SQL database to significantly improve read performance as well as improving the performance of servicing up repeated query results. Mongo is interesting since it means putting a distributed cache in front of your database is not required. Of course, mongo is lacking a lot of sophistication of SQL databases, many of those gaps have to be filled with client code.

A colleague of mine pointed me to this comparison which is a good overview of many of the main No Sql offerings. NoSql is not going to be appropriate for all scenarios, but the great thing is its causing more and more people to question and challenge traditional approaches to building distributed systems.


~ by kevinsmi on October 28, 2012.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: