http://ostatic.com/blog/libres3-open-source-amazon-s3-implementation-released#buzz
Skylable has released LibreS3, an open source implementation of the Amazon S3 service, suitable for installing on private servers in a datacenter. LibreS3 uses Skylable SX, a "reliable, fully distributed cluster solution", on the back end for deduplication and replication. LibreS3 joins a growing list of alternative, open source storage solutions available to the enterprise today.
At first pass, Skylable may seem like another flash in the pan, but the team behind LibreS3 and Skylable SX are open source veterans. The team was the original developers behind ClamAV, one of the most popular anti-virus solutions in the world. The open source pedigree of the Skylable team grants confidence that the team will continue to deliver on a complex and challenging project. Recreating the Amazon S3 service, with most of the API available to third-party tools is no small task, but it is something that the modern datacenter desperately needs.
Once the conversation about a web hosting environment gets to the point where shared, centralized storage is going to be needed, the number of options drops drastically. For Linux, the two main players in the game are either NFS, or CIFS through Samba, and since NFS is natively supported, that's normally where I've seen the system steered. Other options are normally clustered filesystems, which, honestly, I avoid like the plague. Clustered filesystems sound like a great idea, until the daemons you need experience a network hiccup and fence each other off. In the worst of scenarios I've seen, the applications couldn't be started after a reboot because the proprietary clustering software that managed the shared filesystem had decided that the node in question wasn't just rebooted for maintenance, but was malfunctioning, and couldn't be trusted as part of the cluster. Introducing that level of complexity into an already inherently complex system is asking for trouble. NFS, while a much easier and safer solution, also comes with its own level of challenges. NFS can be slow, there are almost always permissions problems, and it can be considered a security risk. Shared filesystems sound like a much better solution than they actually are.
Which is where Skylable and LibreS3 come into play. If you write your application to depend on a call to an S3 bucket to retrieve a file, instead of a call to the local filesystem, you've just opened up your application to a much, much easier path to scaling and upgrading. Your storage can be scaled up completely separately from your application, and your application is no longer reliant on the restrictions inherent in an NFS setup. In my mind, the fact that LibreS3 has a backend storage engine that does dedupication and replication is just the icing on the cake.
I'm not an advocate for throwing new technology into production right away, not without a period of vetting the software first, but LibreS3 has me intrigued. We will be keeping a close eye on this project, and perhaps putting together a small test cluster, as it is shown in this short video, putting together a three node cluster in ten minutes. If you are building a private cloud infrastructure, you may wish to keep tabs on LibreS3 and Skylable as well.
Skylable has released LibreS3, an open source implementation of the Amazon S3 service, suitable for installing on private servers in a datacenter. LibreS3 uses Skylable SX, a "reliable, fully distributed cluster solution", on the back end for deduplication and replication. LibreS3 joins a growing list of alternative, open source storage solutions available to the enterprise today.
At first pass, Skylable may seem like another flash in the pan, but the team behind LibreS3 and Skylable SX are open source veterans. The team was the original developers behind ClamAV, one of the most popular anti-virus solutions in the world. The open source pedigree of the Skylable team grants confidence that the team will continue to deliver on a complex and challenging project. Recreating the Amazon S3 service, with most of the API available to third-party tools is no small task, but it is something that the modern datacenter desperately needs.
Once the conversation about a web hosting environment gets to the point where shared, centralized storage is going to be needed, the number of options drops drastically. For Linux, the two main players in the game are either NFS, or CIFS through Samba, and since NFS is natively supported, that's normally where I've seen the system steered. Other options are normally clustered filesystems, which, honestly, I avoid like the plague. Clustered filesystems sound like a great idea, until the daemons you need experience a network hiccup and fence each other off. In the worst of scenarios I've seen, the applications couldn't be started after a reboot because the proprietary clustering software that managed the shared filesystem had decided that the node in question wasn't just rebooted for maintenance, but was malfunctioning, and couldn't be trusted as part of the cluster. Introducing that level of complexity into an already inherently complex system is asking for trouble. NFS, while a much easier and safer solution, also comes with its own level of challenges. NFS can be slow, there are almost always permissions problems, and it can be considered a security risk. Shared filesystems sound like a much better solution than they actually are.
Which is where Skylable and LibreS3 come into play. If you write your application to depend on a call to an S3 bucket to retrieve a file, instead of a call to the local filesystem, you've just opened up your application to a much, much easier path to scaling and upgrading. Your storage can be scaled up completely separately from your application, and your application is no longer reliant on the restrictions inherent in an NFS setup. In my mind, the fact that LibreS3 has a backend storage engine that does dedupication and replication is just the icing on the cake.
I'm not an advocate for throwing new technology into production right away, not without a period of vetting the software first, but LibreS3 has me intrigued. We will be keeping a close eye on this project, and perhaps putting together a small test cluster, as it is shown in this short video, putting together a three node cluster in ten minutes. If you are building a private cloud infrastructure, you may wish to keep tabs on LibreS3 and Skylable as well.
No comments:
Post a Comment