RAID

Moving Drives on an LSI MegaRAID Controller

Richard Green on

As part of my home lab project which is still on-going (yes, I have been very quiet on this one of late), my plan has been to move my home server into a new chassis to match the other chassis I am using for the home lab. I use Storage Spaces on my home server but as I use an LSI MegaRAID card, I have all of my drives setup as individual RAID0 sets because the MegaRAID family of cards do not support JBOD.

My biggest worry with moving to the new chassis has been the storage and whether or not my RAID0 sets will come across and manage to keep all of my Storage Space data in tact as I have a lot of it and I’d obviously not like to lose it all. This worry is because of the changes in design. My current chassis uses a Reverse Breakout SATA cable to connect the drives, with an SFF-8087 SAS Multilane connection at the controller end and four conventional SATA connections at the drive enclosure end times four for the quad SAS channels on the controller card. The new chassis uses SFF-8087 to SFF-8087 SAS Multilane cables end-to-end and the backplane of the chassis handles the breakout to the individual drives. As a result, I can’t guarantee that all of the drives will be reconnected to the same channel and port on the controller during the move.

I got the chance this week to test this out as I had to add a new 3TB drive to the pool to add capacity. I added the drive to the server, configured a RAID0 for it but I set it up as a Simple NTFS partition disk and not a member of the Storage Space. I put some files on the disk and played around with different options moving the drive around. I pulled the new drive and re-inserted it in a different slot, now connected to a different channel and port but it came up with a foreign configuration on the drive, not exactly what I wanted.

I cleared the foreign configuration on the drive as importing these foreign configurations scares the hell out of me in the event that it overwrites the local configuration and blows away all of the drives including my RAID1 SSD pair for the operating system installation. After clearing the configuration, I created a new RAID0 set and added the drive to it however this time, I did not initialize the drive. After the virtual disk came online, instead of being uninitialized, it came up with the existing formatted partition and my existing test data, good but not perfect as I had to recreate the RAID0 set.

I was looking online for answers and nobody online seems to clearly answer the question how best to move disks around with an LSI MegaRAID controller, strange with these being some of the most dependable and popular cards on the market but I found my answer in the LSI documentation and a little under referenced feature called Drive Roaming.

Drive Roaming occurs when a disk is moved from one port or channel on the controller to another such as my case where I want to move to a new chassis and I am unsure whether all the drives will come back on the same ports and channels. Drive Roaming occurs at start-up when the controller starts up and reads the configuration in NVRAM on the controller and compares it to the configuration stored on each of the drives. This reading of the configuration allows a drive to be moved within the controller to another port or channel and remain configured as before but the emphasis here is on the at start-up.

When I first tested this, I moved the drive online so the drive did not roam, it was pulled and re-inserted. When I did a second test after reading the documentation, I moved the drive with the server shut down and indeed, the drive came back online with the same RAID0 set as configured before and all was working nicely. This is perfect because when I move between the chassis, I will be doing it with the server offline, shutdown and inert so I can now move all of my drives, safe in the knowledge that everything will be retained as was.

Because I know that nothing in IT goes as smoothly as planned however, my fallback option is to re-configure the RAID sets but select the No Initialization option for the drives. To this end, I have recorded the exact configuration of all of my RAID sets. I have recorded the RAID levels, Stripe Sizes, IO and Read Through settings. I have also recorded the current Port and Channel for each drive and the Model and Serial Number for each drive so that I can match the configuration back exactly as it is now. Consistency makes my life easier here. All my drives are 3TB, all Western Digital Caviar Red drives and all are in their own unique RAID0 set and all RAID0 sets are using the same Direct IO and Read Through setting so re-creating the sets is actually a cinch. The only exception is my operating system set which is configured as a RAID1 with two Intel 520 Series SSD drives and I’ve got the settings recorded for them too.

Last but not least, I’ve got my backups. My personal and important data is backed up to Microsoft Azure Backup using the integration with Windows Server 2012 R2 Essentials. To protect the operating system, I’ve got a Windows Backup created locally to a USB hard drive of the operating system partitions so that I can perform a bare metal recovery of the operating system if needed to the RAID1 SSD drives.

This post has been a bit of a waffle and a ramble but I hope that my learning of the LSI MegaRAID Drive Roaming feature helps somebody out there trying to investigate the same thing as me.

Storage Architecture for Windows Server 2012 Essentials

Richard Green on

Two of the best features in my eyes in Windows Server 2012 Essentials over Windows Home Server 2011 are both related to disk.

RAID Support
Windows Server 2012 Essentials is a grown-up Windows Server unlike Windows Home Server 2011 which in an aim to simplify the server in the home idea for consumers, removed the ability to use hardware RAID the operating system volume. This was a horrible thing for Microsoft to do in my opinion.

Storage Spaces
In a nod to Driver Extender from Windows Home Server (v1) Windows 6.2 kernels running on Windows 8 and Windows Server 2012 both support Storage Pools and Storage Spaces. This allows you to pool disks together to produce simple, mirrored or parity volumes from a single pool of disks. It’s like RAID on steroids because it means you only waste the chunks on disk that you want to for volumes that you want to protect, not all of them.

So taking these two ideals into consideration, what am I going to do?

Step 1 is to get the operating system off of the pair of 2TB disks I have, where there is a 60GB partition for the OS and a 1.8TB partition on one disk, and a 1.8TB partition on the second mirrored from the first disk using Windows Disk Management mirroring.

Step 2 is to maximize the utilization of the capacity of my six 2TB disks.

To achieve step 1, I am investing in a pair of SSD disks. For Windows Server 2012 Essentials to accept them they have to be over 160GB, so I am looking at the Intel 520 Series 240GB disks which are currently on Amazon reduced from £300 to £180. These will be connected to my SATA RAID controller in a RAID1 mirror and will be installed in a Lian Li 5.25″ to dual 2.5″ adapter, allowing me to utilize one of the three 5.25″ bays in my case which I will not ever use otherwise, opening up two slots for 3.5″ high capacity disks for future expansion. Needless to say, a pair of Intel 520 Series 240GB disks will give the operating system volume unbelievable IOPS and will allow the server to boot, reboot and access the OS extremely quickly. I’m also going to leave it as one super-sized 240GB partition so that I never have to worry about Windows Updates or software I install on the server causing me to need to think about repartitioning in the future.

To achieve step 2, it’s simple. Connect the now completely free to breath six 2TB disks to any of the on-board or two remaining SATA RAID controller ports, and configure them in Windows Server 2012 Essentials as a single, six disk Storage Pool and carve my volumes out of this 12TB raw disk pool using the protection levels I see fit for my needs.

Thanks to the ability to over provisioning (or thin provisioning as Microsoft incorrectly refer to it in my opinion) on Storage Spaces, I can create spaces larger than my current capacity and add disk or replace existing 2TB disk with 3TB or 4TB disk as available to extend the live capacity.

Over time, as I require more disk there will be one ‘problem’ in that I will have depleted all of my SATA ports. Luckily, my SATA RAID controller supports Port Multipliers, and a cheap and potentially nasty Syba 5 to 1 SATA Port Multiplier for about £45 means I can extend my capability to an extra four ports which at that point reaches the capacity of the case chassis. Power also isn’t an issue as my Corsair AX750 power supply was selected at the time specifically because of it’s amazing ability to run at peak power efficiency at extremely low consumption levels and also to support up to 12 SATA disks with its modular cabling design.

So there we have it…my design for Windows Server 2012 Essentials Storage Architecture. It’s by no means conventional but then I don’t really think anything about my server build is, with it’s 4U rack mount configuration packing a build-out consuming less power than your average light fixture.

I only designed and stood up the Windows Home Server 2011 setup little over a year ago. I think we all secretly knew that Home Server as a product family was a dying breed and that Microsoft would either kill it off completely or encompass it into another product family sooner than later to drop the support overheads. Thankfully it happened sooner I feel: Yes, it means that I have to rebuild my setup not that long after it was actually first built, but thankful because it meant I haven’t invested too heavily in customisation or further expansion of my current setup leaving me playing the corner flag with a legacy product at work. Luckily now, with Windows Server 2012 Essentials being a core SKU in the Windows Server family, it will be three years until the next major release. Although a Windows Server 2012 R2 release may appear sometime in the middle of the three year release cadence for server operating systems, at least being on the RTM release for the same product should make that migration a hell of a lot easier.

A Quick Hit on RAID Levels

Richard Green on

These are written by my own fair hand with nothing copied from Wikipedia, so I accept any errors to be my own.

RAID-0

RAID-0, otherwise known as a Stripe without Parity is a method of pooling two or more disks together into a single logical disk. Although consisting of multiple drives, the OS will only see one disk with the total capacity of all of the provisioned drives. Although attributed as a RAID version, RAID-0 provides no protection for the data and there are no parity bits or duplicate copies of the data on any of the disks – If a single disk fails, all of the data which is stored on that disk is lost and the volume will enter a degraded state.

This level gives the highest value for money as every disk you add grants the full amount of storage, but you get no protection for that money, so it should not be used by itself as a data protection solution.

RAID-1

Otherwise known as Mirroring, RAID-1 provides the simplest level of data protection. Two disks are mirrored, so that they are exact copies of each other. In the event of a single drive failure, the other drive takes up the role as the single serviceable disk and performs all of the I/O. Until the failed disk is replaced however, the data is at risk as should the second disk fail then you will have no more copies of the data.

RAID-5

The most commonly used RAID method in older business systems, RAID-5, otherwise known as stripe with parity, requires a minimum of three disks. The storage capacity of the array equals the capacity of two of the disks. The capacity equal to the third disk is used to store the parity data. Two of the three disks can simultaneously fail in RAID-5 and the parity data on the last remaining disk is able to recreate the existing data to it’s full healthy state.

Read performance on RAID-5 is good, however write performance is slow due to the overhead to write the parity data, especially on low grade performance on-board RAID controllers seen on modern day motherboards.

RAID-10

My personal favourite, RAID-10 is what is known as a hybrid RAID or nested RAID technology. It uses a recommended minimum of four disks, although in some configurations can work with two. RAID-10 is actually correctly noted as RAID-1+0. This means RAID-1 combined with RAID-0. A stripe without parity (RAID-0) offers good read and write performance, whilst offering maximum storage capacity, but it offers no protection. RAID-1 Mirror offers good performance for both reads and writes, but the performance is below that of RAID-0. RAID-1 provides the protection of a second copy of all of the data.

RAID-10 fixes this by backing each striped disk with a mirrored copy. In a four disk RAID-10 configuration, the capacity of the volume is half that of the disks total (Eg. 4x 2TB disks gives a physical disk size of 8TB, and a usable disk size of 4TB).

RAID-10 gives you the performance of RAID-0 plus the added performance benefit of having a second disk spindle hitting each I/O, in theory halving the seek time to read or write data to the disk, plus the protection benefit of a second copy of all of the data should one drive fail. In RAID-10, two of the four drives can fail at any one time, however there must be one drive from each Mirror set available to complete the Stripe.