Monday, March 25, 2013

Amazon Cloud Storage Clone Goes Open Source

Amazon’s cloud services are a great thing. They give you instant access to computing power over the web, letting you instantly store data and run applications. But if you want to set up your own cloud service, Amazon can’t help you. You need software like Riak CS.
Riak CS is a file system for storing data, and it’s designed to be fully compatible with Amazon’s popular cloud storage service, S3. It can be used to create private Amazon-style storage systems in your own data center, to build public services that compete with Amazon, or to power web applications of your own creation. And as of Wednesday, it’s open source.
It’s yet another example of an Amazon mimic that’s freely available for anyone to use on their own. This includes everything from the OpenStack project to various NoSQL databases.
Riak CS started life as a proprietary version of the open source NoSQL database Riak. But unlike many other “open core” systems, Riak CS wasn’t a souped up version of the open source version, but a purpose-built application with new features specifically designed for large scale storage. Riak CS added support for “objects” — i.e., files — of up to 5 GB each, and tools for billing individual users or departments for resource use. And of course, an S3-compatible API that let you plug applications into it in much the same way you’d plug them into Amazon’s online service.

In addition to open sourcing the code to Riak CS, the company has added several new features, including the ability to split large uploads into smaller parts and a new graphical, web-based admin console.
But why go open source now, almost a year after the product was first released? “It’s easier to go in one direction, and not the other, we wanted to make sure we did it right,” says Justin Sheehy, the CTO of Basho, the company that develops Riak. “It was a brand new product for us.”
“Over the past year we’ve had an opportunity to learn more from our customers,” adds Shanley Kane, a product manager at Basho.
Curious developers are also often more attracted to open source technologies than proprietary ones. Sheehy says he’s happy with the current traction the product has, but thinks going open source will make it happen quicker. It will also make integrations much easier.
Basho is hoping to be a part of the large cloud ecosystem. The open source cloud building system OpenStack already has a dedicated open source storage layer — the open source Swift — but it’s no secret that Basho would like to provide an alternative. And Basho has already partnered with Citrix to provide integration between Riak CS the Citrix’s open source cloud system CloudStack.
But Basho will still offer an enterprise version of Riak CS that adds support for multi-data center replication. But Sheehy stresses that the open source version is viable without this feature. In fact, he says, Amazon S3, doesn’t actually support replication on its own between different regions.


  1. This comment has been removed by the author.

  2. This comment has been removed by the author.

  3. Thanks for sharing the post related to S3 Compatible Storage which is highly informative. S3 Compatible Storage enables service providers and enterprises to build reliable, affordable and scalable cloud storage solutions.