http://www.makeuseof.com/tag/how-to-enable-storage-pooling-and-mirroring-using-btrfs-for-linux
If you have multiple hard drives in your Linux system, you don’t have to treat them all as different storage devices. With Btrfs, you can very easily create a storage pool out of those hard drives.
Under certain conditions, you can even enable mirroring so you won’t lose your data due to hard drive failure. With everything set up, you can just throw whatever you want into the pool and make the most use of the storage space you have.
There isn’t a GUI configuration utility that can make all of this easier (yet), but it’s still pretty easy to do with the command line. I’ll walk you through a simple setup for using several hard drives together.
If you’re not sure what a filesystem is, take a look at this explanation of a few filesystems for Windows. You can also check out this nice comparison of various filesystems to get a better idea of the differences between existing filesystems.
Btrfs is still considered “not stable” by many, but most features are already stable enough for personal use — it’s only a few select features where you might encounter some unintended results.
While Btrfs aims to be the default filesystem for Linux at some point in the future, it’s still best to use ext4 for single hard drive setups or for setups that don’t need storage pooling and mirroring.
You can also assume that you have another hard drive (/dev/sda) of some arbitrary size which contains your bootloader and operating system. We’re not concerning ourselves about /dev/sda and are solely combining the other four hard drives for extra storage purposes.
To create a Btrfs filesystem on one of your hard drives, you can use the command:
Of course, you can replace /dev/sdb with the actual hard drive you want to use. From here, you can add other hard drives to the Btrfs system to make it one single partition that spans across all hard drives that you add. First, mount the first Btrfs hard drive using the command:
Then, run the commands:
Now, you can add them to the first hard drive using the commands:
If you had some data stored on the first hard drive, you’ll want the filesystem to balance it out among all of the newly added hard drives. You can do this with the command:
Alternatively, if you know before you even begin that you want a Btrfs filesystem to span across all hard drives, you can simply run the command:
Of course this is much easier, but you’ll need to use the method mentioned above if you don’t add them all in one go.
You’ll notice that I used a flag: “-d single”. This is necessary because I wanted a RAID 0 configuration (where the data is split among all the hard drives but no mirroring occurs), but the “single” profile is needed when the hard drives are different sizes. If all hard drives were the same size, I could instead use the flag “-d raid0″. The “-d” flag, by the way, stands for data and allows you to specify the data configuration you want. There’s also an “-m” flag which does the exact same thing for metadata.
Besides this, you can also enable RAID 1 using “-d raid1″ which will duplicate data across all devices, so using this flag during the creation of the Btrfs filesystem that spans all hard drives would mean that you only get 500GB of usable space, as the three other hard drives are used for mirroring.
Lastly, you can enable RAID 10 using “-d raid10″. This will do a mix of both RAID 0 and RAID 1, so it’ll give you 1.5TB of usable space as the two 1TB hard drives are paired in mirroring and the two 500GB hard drives are paired in mirroring.
If you have a Btrfs filesystem that you’d like to convert to a different RAID configuration, that’s easily done. First, mount the filesystem (if it isn’t already) using the command:sudo
Then, run the command:
sudo
This will change the configuration to RAID 1, but you can replace that with whatever configuration you want (so long as it’s actually allowed — for example, you can’t switch to RAID 10 if you don’t have at least four hard drives). Additionally, the -mconvert flag is optional if you’re just concerned about the data but not the metadata.
Then fix the filesystem with:
If you didn’t have RAID 1 or RAID 10 enabled, any data that was on the failed hard drive is now lost.
Of course, replace /dev/sdc with the hard drive you want to remove. This command will take some time because it needs to move all of the data off the hard drive being removed, and will likewise fail if there’s not enough room on the other remaining hard drives.
If you want the Btrfs filesystem to be mounted automatically, you can place this into your /etc/fstab file:
What’s the biggest storage pool you could make? Do you think storage pools are worthwhile? Let us know in the comments!
If you have multiple hard drives in your Linux system, you don’t have to treat them all as different storage devices. With Btrfs, you can very easily create a storage pool out of those hard drives.
Under certain conditions, you can even enable mirroring so you won’t lose your data due to hard drive failure. With everything set up, you can just throw whatever you want into the pool and make the most use of the storage space you have.
There isn’t a GUI configuration utility that can make all of this easier (yet), but it’s still pretty easy to do with the command line. I’ll walk you through a simple setup for using several hard drives together.
What’s Btrfs?
Btrfs (called B-tree filesystem, “Butter FS”, or “Better FS”) is an upcoming filesystem that incorporates many different features at the filesystem level normally only available as separate software packages. While Btrfs has many noteworthy features (such as filesystem snapshots), the two we’re going to take a look at in this article are storage pooling and mirroring.If you’re not sure what a filesystem is, take a look at this explanation of a few filesystems for Windows. You can also check out this nice comparison of various filesystems to get a better idea of the differences between existing filesystems.
Btrfs is still considered “not stable” by many, but most features are already stable enough for personal use — it’s only a few select features where you might encounter some unintended results.
While Btrfs aims to be the default filesystem for Linux at some point in the future, it’s still best to use ext4 for single hard drive setups or for setups that don’t need storage pooling and mirroring.
Pooling Your Drives
For this example, we’re going to use a four hard drive setup. There are two hard drives (/dev/sdb and /dev/sdc) with 1TB each, and two other hard drives (/dev/sdd and /dev/sde) with 500GB for a total of four hard drives with a total of 3TB of storage.You can also assume that you have another hard drive (/dev/sda) of some arbitrary size which contains your bootloader and operating system. We’re not concerning ourselves about /dev/sda and are solely combining the other four hard drives for extra storage purposes.
Creating A Filesystem
To create a Btrfs filesystem on one of your hard drives, you can use the command:
sudo mkfs.btrfs /dev/sdbOf course, you can replace /dev/sdb with the actual hard drive you want to use. From here, you can add other hard drives to the Btrfs system to make it one single partition that spans across all hard drives that you add. First, mount the first Btrfs hard drive using the command:
sudo mount /dev/sdb /mntThen, run the commands:
sudo mkfs.btrfs /dev/sdc mkfs.btrfs /dev/sdd mkfs.btrfs /dev/sdeNow, you can add them to the first hard drive using the commands:
sudo btrfs device add /dev/sdc /mnt btrfs device add /dev/sdd /mnt btrfs device add /dev/sde /mntIf you had some data stored on the first hard drive, you’ll want the filesystem to balance it out among all of the newly added hard drives. You can do this with the command:
sudo btrfs filesystem balance /mntAlternatively, if you know before you even begin that you want a Btrfs filesystem to span across all hard drives, you can simply run the command:
sudo mkfs.btrfs -d single /dev/sdb /dev/sdc /dev/sdd /dev/sdeOf course this is much easier, but you’ll need to use the method mentioned above if you don’t add them all in one go.
You’ll notice that I used a flag: “-d single”. This is necessary because I wanted a RAID 0 configuration (where the data is split among all the hard drives but no mirroring occurs), but the “single” profile is needed when the hard drives are different sizes. If all hard drives were the same size, I could instead use the flag “-d raid0″. The “-d” flag, by the way, stands for data and allows you to specify the data configuration you want. There’s also an “-m” flag which does the exact same thing for metadata.
Besides this, you can also enable RAID 1 using “-d raid1″ which will duplicate data across all devices, so using this flag during the creation of the Btrfs filesystem that spans all hard drives would mean that you only get 500GB of usable space, as the three other hard drives are used for mirroring.
Lastly, you can enable RAID 10 using “-d raid10″. This will do a mix of both RAID 0 and RAID 1, so it’ll give you 1.5TB of usable space as the two 1TB hard drives are paired in mirroring and the two 500GB hard drives are paired in mirroring.
Converting A Filesystem
If you have a Btrfs filesystem that you’d like to convert to a different RAID configuration, that’s easily done. First, mount the filesystem (if it isn’t already) using the command:sudo
mount /dev/sdb1 /mntThen, run the command:
sudo
btrfs balance start -dconvert=raid1 -mconvert=raid1 /mntThis will change the configuration to RAID 1, but you can replace that with whatever configuration you want (so long as it’s actually allowed — for example, you can’t switch to RAID 10 if you don’t have at least four hard drives). Additionally, the -mconvert flag is optional if you’re just concerned about the data but not the metadata.
If Hard Drive Failure Occurs
If a hard drive fails, you’ll need to remove it from the filesystem so the rest of the pooled drives will work properly. Mount the filesystem with the command:sudo mount -o degraded /dev/sdb /mntThen fix the filesystem with:
sudo btrfs device delete missing /mntIf you didn’t have RAID 1 or RAID 10 enabled, any data that was on the failed hard drive is now lost.
Removing A Hard Drive From The Filesystem
Finally, if you want to remove a device from a Btrfs filesystem, and the filesystem is mounted to /mnt, you can do so with the command:sudo btrfs device delete /dev/sdc /mntOf course, replace /dev/sdc with the hard drive you want to remove. This command will take some time because it needs to move all of the data off the hard drive being removed, and will likewise fail if there’s not enough room on the other remaining hard drives.
Automatic Mounting
If you want the Btrfs filesystem to be mounted automatically, you can place this into your /etc/fstab file:
sudo /dev/sdb /mnt btrfs device=/dev/sdb,device=/dev/sdc,device=/dev/sdd,device=/dev/sde 0 0Mount Options
One more bonus tip! You can optimize Btrfs’s performance in your /etc/fstab file under the mount options for the Btrfs filesystem. For large storage arrays, these options are best:compress-force=zlib,autodefrag,nospace_cache.
Specifically, compress=zlib will compress all the data so that you can
make the most use of the storage space you have. For the record, SSD
users can use these options: noatime,compress=lzo,ssd,discard,space_cache,autodefrag,inode_cache. These options go right along with the device specifications, so a complete line in /etc/fstab for SSD users would look like:sudo /dev/sdb /mnt btrfs device=/dev/sdb,device=/dev/sdc,device=/dev/sdd,device=/dev/sde,
noatime,compress=lzo,ssd,discard,space_cache,autodefrag,inode_cache 0 0How Big Is Your Storage Pool?
Btrfs is a fantastic option for storage pooling and mirroring that is sure to become more popular once it is deemed completely stable. It also wouldn’t hurt for there to be a GUI to make configuration easier (besides in some distribution installers), but the commands you have to use in the terminal are easy to grasp and apply.What’s the biggest storage pool you could make? Do you think storage pools are worthwhile? Let us know in the comments!
No comments:
Post a Comment