storage pool

DPM Replica Recovery Point Run Out of Space

Richard Green on

If you receive an alert in System Center Data Protection Manager (DPM) that a replica or recovery point volume has run out of space, you will probably find this is a result of your DPM Storage Pool being out of space and head off to talk to your storage administrator to get some additional disk presented. While this is obviously the correct thing to do, you also need to take into consideration the impact this may have on your Recovery Point and Replica volumes.

Once you have added a new disk to the server, you will add it to the DPM Storage Pool to extend the capacity of the pool. With the pool extended, it would be logical to assume that DPM will automatically extended the Replica and Recovery Point volumes and whilst this is true for normal operation that DPM will auto-grow these volumes if you have enabled the auto-grow feature, if you’re DPM Storage Pool completely filled before the new disk was added, you will need to do this manually. When DPM has attempted to auto-grow the partition on a previous attempt but been unable to do so due to insufficient disk space, it puts the Protection Group into a state where this operation is not attempted again automatically.

Imagine a scenario whereby a DPM server has a single 2TB volume in use for the Storage Pool. DPM creates many dynamic partitions on the disk to store your Protection Group data. When the disk fills and DPM needs to start using a new disk that you added, it will convert the existing dynamic partition into a spanned partition to allow it to span the multiple Storage Pool disks. If this operation occurs during normal DPM operation whereby there is sufficient free space to do so then it will happen automatically and you have nothing to worry about. If however, your DPM Storage Pool completely fills before it has a chance to convert it to a spanned partition, even once their capacity, DPM will stop trying to perform this operation.

Luckily for us, the fix is pretty simple. Locate the Protection Group which is reporting that the Replica or Recovery Point volumes are out of space and extend them manually using the DPM Central Console by any amount you like. It can be as little as 100MB if you need it to be. This manual extension will force DPM to read the data from the Windows Logical Disk Manager (LDM), it will see the new disk available and perform the span conversion operation.

If you can’t identify via the DPM Central Console which Protection Groups are faulting, another way is to look at the Disk Management console and look for partitions on the DPM Storage Pool disks which are not of type Spanned. Non-spanned partitions in this instance will be partitions which have not been pulled across both disks. This could be because that Protection Group hasn’t yet needed to be extended to make use of the new disk or it could be because it’s out of space but it’s a step in the right direct.

If you are extending the existing DPM Storage Pool disk instead of adding a new disk, I’m not exactly sure what would happen. If I had to hazard a guess, I would probably say that DPM will know about this uplift in capacity and extend the Protection Groups automatically as you are still working with the same disk, therefore no span conversion operation is required however I could be wrong? This is something for me to test in my new lab once I get it built.

Storage Spaces Inaccessible After Windows Server 2012 R2 Upgrade

Richard Green on

Windows Server 2012 R2 has some nice new features and improvements on existing features for users of Storage Spaces so there is a definite appeal for users of Windows Server 2012 to want to upgrade to Windows Server 2012 R2.  If you opt to do an in-place upgrade to preserve your existing Storage Spaces so that you can get your service up and running with the hope of being able to use them straight off the bat in Windows Server 2012 R2, you may encounter an error Read-only by user action and you need to perform some corrective steps to use them again.

Storage Space Read-Only User Action

This is what your Storage Spaces may look like if you open the Storage Spaces control panel item after the upgrade. As you can see, the spaces are in-tact and will report all of the space names and capacity from prior to the upgrade but instead of being online as you are used to seeing, you instead have this information icon and a message alongside the pool capacity indicator Read-only by user action. This is a built in protection feature of Windows Server 2012 R2 which takes your Storage Spaces offline by default after an upgrade. We just simply need to bring them online to use them. This is very similar to how in Windows Server 2003, a disk connected from an external system from a software RAID set could be marked as Foreign and the configuration of the disk needs to be imported first.

Changing the Storage Pool Status to Read/Write

To do this, open an administrative PowerShell prompt. At PowerShell, enter the two Cmdlets as follows:

Get-StoragePool | Where-Object {$_.IsReadOnly -eq $True} | Set-StoragePool -IsReadOnly $False
Get-VirtualDisk | Where-Object {$_.IsManualAttach -eq $True} | Set-VirtualDisk -IsManualAttach $False

If you forget to elevate the PowerShell prompt by running it as an administrator you will get access denied responses to the two Cmdlets as you aren’t running the Cmdlets with your administrative rights. Simply close PowerShell and re-open it by right-clicking and using the Run as Administrator option.

Bring the Storage Spaces Online

Once you’ve entered the Cmdlets above, returning to the Storage Spaces control panel applet now, you will see the information shown has updated.

Storage Space Offline by Policy

As you can see, the Storage Spaces are now reporting their status as OK but they are marked as Offline by Policy. To change this and to bring the Storage Spaces online, simply click the Bring Online option next to each Storage Space and it will be brought online and granted a drive letter.

Check, Verify and Reminder

It’s important to note here that the drive letter assigned will be the next free letter and not perhaps, the drive letter that you used on the previous installation of Windows Server 2012. If you have a requirement for the Storage Space to be on a particular letter then you will need to go into the Properties of the individual spaces after it has been brought online and change the letter.

It’s also good to remember that any file shares you had on these Storage Spaces may be un-shared through the upgrade process so you should check the existence of your shares either by using the Properties on the drive or folder which you needed to be shared or by using the Share and Storage Management administrative console.

Once you’ve got your Storage Spaces all brought online after the actions above, you should be looking like normality again as shown below.

Storage Space Online Normal

Hopefully someone out there finds this useful and it saves at least a few hair extractions from taking place after a Windows Server 2012 to Windows Server 2012 R2 in-place upgrade. Now it’s time to go and enjoy those new features.

The Problem with Storage Spaces

Richard Green on

As you may well have gathered from a number of my previous posts about Windows Server 2012 and Storage Pools, I was intending on using them for my home server rebuild, and I am indeed using them, however I have neglected to post anything showing the new server (although I will change that shortly).

I ran into a problem with Storage Pools today which I think quite frankly blows. I got myself a new Western Digital 3TB Red drive to try out. The plan is to replace all of my existing six 2TB Western Digital Green drives with these for a number of reasons including greater bang for buck on power consumption, increased IOPS, cooler running temperature and improved reliability.

Not wanting to keep a mixture of Green and Red drives for very long, I proceeded to remove one of the drives from the pool to replace with a Red drive. The Storage Pool refused to remove it as a Simple non-redundant Storage Space was being hosted on this drive.

Problem 1:  Storage Spaces cannot be converted between Simple, Mirror or Parity. Once they are created, they are created. My only option for this was to create a new temporary Space marked as Mirror and copy the data from the Simple so that I could delete it. Once deleted, I tried a second attempt to remove the drive and I got an error that I needed to add another drive to the pool as there was insufficient capacity.

I’m sorry, what?

Problem 2: I have six 2TB drives in an uber-pool. I am currently less than half of it, so removing the drive should be no problem. I tried this a few more times and each time I got the same error that I would need to add more capacity to the pool before I would be able to remove the drive, which I know to be cobblers.

In the end, I just pulled the disk from the server and let the Storage Pool have a cry about the missing disk. From here, I marked the disk for removal to allow Windows to think that the disk was failed and that it was never coming back. This worked although is time consuming as it forces all Mirror and Parity virtual disks to enter a repairing state, copying blocks to remaining disks in the pool to keep up the protection level.

This brings me softly onto another point which is more of a beef.

Beef 1: One of the tricks of Windows Server 2012 was deduplication. Anyone familiar with Windows Server 2012 will know that Storage Pools and deduplication do work together, but in Essentials, deduplication is absent, missing, not there. The feature is completely missing from any of the Server Manager interfaces and from PowerShell, the command Get-Command -Module Dedup* returns nothing.

Why is it missing from Essentials? Essentials is the release of Windows Server 2012 targeted at SMB/SME and pro-home customers, the customers most likely to be storing a lot of data on a tight budget, so why strip out the feature that they will probably be highly interested in, in Windows Server 2012.

I really hope that Microsoft get enough complaints from customers of Essentials to release a Feature Update to re-add the support for deduplication.

With this done,