Essentials

Project Home Lab: Open Server Surgery

Richard Green on

So with my recent bought of activity on the home lab project front, evident from my previous posts Project Home Lab: Server Build and Project Home Lab: Servers Built, I’ve forged ahead and got some of the really challenging an blocking bits of the project done over Christmas. I eluded to what I needed to do next in the post Project Home Lab: Servers Built. All of this work paves the way for me to get the project completed in good order, hopefully all during January, at long last.

In this post I’m just going to gloss over some of the details about the home server move that I completed over the weekend. Lots more hours of thinking, note taking and planning were involved in this and most likely more than should have gone into it but I don’t like failure so I like to make sure I have all the bases covered and all the boxes ticked. Most critically, I had to arrange an outage window and downtime with the wife for this to happen.

Out with the Old

The now previous incarnation of my Windows Server 2012 R2 Essentials home server lived in a 4U rack mount chassis. As it was the only server I possessed at the time, I never bothered with rack mount rails so problem one was in the fact that the server was just resting atop of my UPS in the bottom of the rack.

Problem two and luckily, something which has never bitten me previously but has long bothered me is that the server ran on desktop parts inside a server chassis. As a result, it had no IPMI interface for out of band management so that if something should go wrong with the Windows install or some warning in the BIOS, I can remotely access the keyboard, video and mouse (a KVM no less). It had an Intel Core i5 3740T processor with an ASUS ATX motherboard and unregistered unbuffered memory with a desktop power supply albeit it a high quality Corsair one. All good quality hardware but not optimal for a server build.

The biggest problem however was with the fact that the 4U chassis, a previous purchase from X-Case a couple of years ago sat at 4U tall but only had capacity for ten external disks. I had two 2.5″ SSDs for the operating system mounted internally in one of the 5.25″ bays in a dual 2.5″ drive adapter in addition to the external drives. It all worked nicely but it wasn’t ideal as my storage needs are growing and I only had two free slots. Although not a problem, the hot swap drive bays were added to the chassis with an aftermarket upgrade from X-Case didn’t use SAS Multilane SFF-8087 cables but instead used SATA connections which meant from my LSI 9280-16i4e RAID Controller, I had to use SAS to SATA Reverse Fanout cables which made the whole affair a bit untidy.

None of this is X-Case’s fault let us remember. The case did it’s job very well but my evolving and increasingly demanding needs no longer met the capabilities of the case.

Planning for the New

Because I like their to be order in the force, per my shopping list at Project Home Lab: Shopping List, I bought a new 3U X-Case chassis for my home server at the same time as buying up the lab components and getting the home server set straight is priority one because the 4U chassis is a blocker to me getting any further work done as the 3U and 2U lab servers need to fit in above it. In addition to moving chassis, I’ve given it an overhaul with a new motherboard and CPU to match the hardware in the lab environment. A smaller catalogue of parts means less knowledge required to maintain the environments and means I have an easy way of upgrading or retro-fitting in the future with the single design ethos.

As anyone knows, changing the motherboard, processor and all of the underlying system components in a Windows server is a nightmare potentially in the making so I had to plan well for this.

I had meticulously noted all of the drive configurations from the RAID Controller down to the last detail, I had noted which drives connected to which SATA port on which controller port and I had a full backup of the system state to perform a bare metal recovery if I needed. All of our user data is backed up to Azure so that I can restore it if needed although I didn’t expect any problems with the data drives in honesty, it was the operating system drives I was most concerned about.

In with the New

After getting approval for the service outage from the wife and shutting down the old home server, I got it all disconnected and removed from the rack. I began the painful process of unscrewing all of my eight drives from the old chassis drive caddy’s and the two internal drives and reinstalling them into the new caddy’s using the 2.5″ to 3.5″ adapters from the shopping list. I think I probably spent about 45 minutes carefully screwing and unscrewing drives and at the same time, noting which slot I removed them from and which slot I installed them into.

With all the drives moved over, I moved over the RAID Controller and connected up the SAS Multilane SFF-8087 cables to the connector with the tail end already connected to the storage backplanes in the chassis.

Once finished, I connected up the power and the IPMI network port on the home server which I had already configured with a static IP as the home server is my DHCP Server so it wouldn’t be able to get an automatic lease address. I got connected to the IPMI interface okay and powered the server on using it and quickly flipped over to the Remote Control mode which I have to say, works really nicely even when you consider that it’s Java based.

Up with the New

While I was building the chassis for the home server, I had already done some of the pre-work to minimize the downtime. The BIOS was already upgraded to the latest version along with the on-board SAS2008 controller and the IPMI firmware. I had also already configured all of the BIOS options for AHCI and a few other bits (I’ll give out all of the technicalities of this in another post later).

First things first, the Drive Roaming feature on the LSI controller which I blogged about previously, Moving Drives on an LSI MegaRAID Controller worked perfectly. All 9 of the virtual drives on the controller were detected correctly, the RAID1 Mirror for the OS drives stayed in-tact and I knew that the first major hurdle was a behind me. A problem here would have been the most significant to timely progress.

The boot drive was hit okay from the LSI RAID Controller BIOS and the Windows Server 2012 R2 logo appeared at least showing me that it was starting to do something. It hung here for a couple of minutes and then the words “Getting Devices Ready” appeared. The server hung here for at least another 10 minutes at which point I was starting to get worried. Just when I was thinking about powering it off and moving all the drives back and reverting my changes, a percentage appeared after the words “Getting Devices Ready”, starting at 35% and it quickly soared up to 100% and the server rebooted.

After this reboot, the server booted normally into Windows. It took me about another hour after this to clean-up the server. First I had to reconfigure my network adapter team to include the two on-board Gigabit Ethernet adapters on the Supermicro motherboard as I am no longer using the Intel PCIe adapter from the old chassis. Then, using the set devmgr_show_nonpresent_devices=1 trick, I removed all of the references to and uninstalled the drivers for the old server hardware.

After another reboot or two to make sure everything was working properly and a thorough check of the event logs for any Active Directory, DNS or DHCP errors and a test from my 3G smartphone to make sure that my published website was running okay on the new server, I called it a success. One thing I noted of interested here was that Windows appeared to not require re-activation as I had suspected it would. A motherboard and CPU family change would be considered a major hardware update which normally requires re-activation of the license key but even checking now, it reports activated.

Here’s some Mr. Blurrycam shots of the old 4U chassis after I removed it and the new 3U chassis in the rack.

WP_20141230_009

WP_20141230_006

As you can see from the second picture, the bottom 3U chassis is powered up and this is the home server. In disk slots 1 and 5 I have the two Intel 520 Series SSDs which make up the operating system RAID1 Mirror and in the remaining eight populated slots are all 3TB Western Digital Red drives.

Above the home server is the other 3U chassis which will be the Lab Storage Server once all is said and done and at the very bottom I have the APC 1500VA UPS which is quite happy at 20% load running the home server along with my switches, firewall and access points via PoE. I’ll post some proper pictures of the rack once everything is finished.

Behind the scenes, I had to do some cabling in the back of the rack to add a new cable for the home server IPMI interface which I didn’t have before and the existing cables for the home server NIC Team were a bit too tight for my liking caused by the 3U Lab Storage Server above being quite deep and pulling on them slightly. To fix this, I’ve patched up two new cables of longer length and routed them properly in the rack. I’ve got a lot of cables to make soon for the lab (14 no less) and I will be doing some better cable management at the same time as that job. One of the nice touches on the new X-Case RM316 Pro chassis is the front indicators for the network ports, both of which light up and work with the Supermicro on-board Intel Gigabit Ethernet ports. The fanatic in me wishes they were blue LEDs for the network lights to match the power and drive lights but that’s not really important now is it.

More Home Server Changes

The home server has now been running for two days without so much as a hiccup or a cough. I’m keeping an eye on the event logs in Windows and the IPMI alarms and sensor readings in the bedding in period and it all looks really happy.

To say thank you to the home server for playing so nicely during it’s open server surgery, I’ve got three new Western Digital 5TB drives to feed it some extra storage. Two of the existing 3TB drives will be coming out to make up the bulk storage portion of the Lab Storage Server Storage Space and one drive will be an expansion giving me a gross uplift of 9TB capacity in the pool. I would be exchanging the 3TB drives in the home server with larger capacity drives one day in the future anyway so I figured I may as well do two of them early and make good use of the old drives for the lab.

I’m also exploring the options of following the TechNet documentation for transitioning from Windows Server 2012 R2 Essentials to Windows Server 2012 R2 Standard. You still get all of the Essentials features but on a mainline SKU which means less potential issues with software (like System Center Endpoint Protection for example which won’t install on Essentials). On this point I’m waiting for some confirmation of the transition behaviour in a TechNet Forum question I raised at https://social.technet.microsoft.com/Forums/en-US/d888f37a-e9e9-4553-b24c-ebf0b845aaf1/office-365-features-after-windows-server-standard-transition?forum=winserveressentials&prof=required as the TechNet article at http://technet.microsoft.com/en-us/library/jj247582 leaves a little to be desired in terms of information.

I’m debating buying up some Lindy USB Port Blockers (http://www.lindy.co.uk/accessories-c9/security-c388/usb-rj-45-port-blockers-locks-c390/usb-port-blocker-pack-of-4-colour-code-blue-p2324) for the front access USB ports on all the servers so that it won’t be possible for anyone to insert anything in the ports without the unlocking tool to open up the port first. See if you can guess which colour I might buy?

Up Next

Next on my to do list is the re-addressing of the network, breaking out my hacksaw and cabling.

The re-addressing of the network is make room for the new VLANs and associated addressing which I will be adding for the lab and my new addressing schema makes it much easier for me longer term to manage. This is going to be a difficult job much like the job I’ve just finished. I’ve got a bit of planning to finish for this before I can do it so this probably won’t happen now until after new year.

The hacksaw, as drastic as that sounds is for the 2U Hyper-V server which you may notice is not racked in the picture above. For some reason, the sliding rails for the 2U chassis are too long for my rack and with the server installed on the rails and pushed back, it sits about an inch and half proud of the posts which no only means I can’t screw it down in place but I can’t close the rack door. I’m going to be hacking about two inches off the end of the rails so that I can get the server to sit flush in the rack. It’s only a small job but I need to measure twice and cut once as my Dad taught me.

As I mentioned before, I’ve got some 14 cables I need to make and test for the lab and this is something I can be working on in parallel to the other activities so I’m going to be trying to make a start on these soon so that once I have the 2U rails cut to size correctly, I can cable up at the same time.

Windows Server 2012 Essentials Folder Redirection on Windows 8.1

Richard Green on

As all good IT Pros have done, I’ve upgraded my home client computers from Windows 8 to Windows 8.1. You have upgraded your machines to Windows 8.1 right?

As I frequently proclaim and preach on here, I run Windows Server 2012 Essentials on my home network, acting as my DNS Server, DHCP Server in addition to the out of the box features that you can get from Windows Server 2012 Essentials like roaming profiles, folder redirection, automated computer backups and network file sharing (all of which I use).

When I was building out a test environment this week to practice how I might migrate from Windows Server 2012 Essentials to Windows Server 2012 R2 Essentials without the benefit of a second server with 19TB of available storage to hand (how many homes do have 19TB of storage let alone a spare 19TB) I was experiencing an issue.

As part of my testing, I built a Windows 8.1 Pro virtual machine to simulate a desktop or laptop client computer. I built a Windows Server 2012 Essentials server as a second virtual machine on which I recreated my group policy settings and a mock up of my Storage Pool and Storage Spaces on my production server. After installing the Windows Server 2012 Essentials Connector on the Windows 8.1 client and logging in for the first time as a user configured to use roaming profile and folder redirection, I noticed that the roaming profile was working but the folder redirection was not.

I spent a while pouring through event logs on the client wondering why folder redirection wasn’t working, looking at GPMC (Group Policy Management Console) wondering if I’d done something silly like moved a link on a GPO preventing it from working until the penny dropped. Windows Server 2012 Essentials applies a WMI Filter named SBS Group Policy WMI Filter to the SBS Group Policy Folder Redirection GPO which is created when you implement Group Policy via the Server Dashboard.

Windows Server 2012 Essentials Original WMI Filter

This WMI Filter is setup as SELECT * FROM Win32_OperatingSystem WHERE (Version LIKE “6.1%” or Version LIKE “6.2%”) AND ProductType = “1”. For those who are now also dropping the penny or those who can’t make head nor tail of a WMI Filter, Windows 8.1 increments the operating system version number from 6.2 (Windows 8) to 6.3 (Windows 8.1), therefore the GPO isn’t applying to any of the Windows 8.1 machines on my network because this filter limits the scope of the Group Policy Object to explicitly Windows 7 and Windows 8 operating systems.

The solution to making this work is pretty simple in that we just need to update the WMI Filter so that it includes Windows 8.1 as we know that basic features like roaming profiles and folder redirection are going to work so I’m not worried about something breaking here.

I’ve decided to change my WMI Filter to include operating systems greater than or equal to Windows 7 rather than add another or statement to include Windows 8.1 For me, the WMI Filter now reads SELECT * FROM Win32_OperatingSystem WHERE (Version >= “6.1%”) AND ProductType = “1”.

Windows Server 2012 Essentials New WMI Filter

 

After making the changes and running a gpupdate command on a Windows 8.1 client computer, the group policy magically springs back into life and things start working. Firstly, I’m amazed that I haven’t noticed this being a problem on my home clients which I guess is a testament to my gigabit throughout home network pushing the files directly back to the server rather than caching them locally with Offline Folders first. Secondly, I’m surprised that this hasn’t been updated with a patch or update to Windows Server 2012 Essentials but perhaps this is a cattle prod for customers to upgrade to Windows Server 2012 R2 Essentials?

Deduplication in Windows Server 2012 Essentials

Richard Green on

Yesterday, I posted with a quasi-rant about Windows Server 2012 Essentials Storage Pools and the inability to remove a disk in a sensible non-destructive manner. At the end of that post, I eluded to the lack of the Primary Data Deduplication feature in Windows Server 2012 Essentials which got me thinking about it more, so I went of on an internet duck hunt to find the solution.

Firstly, I found this thread (http://social.technet.microsoft.com/Forums/en-US/winserveressentials/thread/4288f259-cf87-4bd6-bf9f-babfe26b5a69) on the TechNet forums in which an MVP highlights a bug which was filed on Microsoft Connect during the beta stages over the lack of deduplication. The bug was closed by Microsoft with a status of ‘Postponed’ and a message that it was a business decision to remove the feature.

Sad, but true when the people being targeted with Essentials are the people potentially wanting and needing it most, but I guess the reason probably lies in the realms of supportability and a degree of knowledge gap in the home and small business sectors to understand the feature.

Luckily for me, in another search, I found this article (http://forums.mydigitallife.info/archive/index.php/t-34417.html) at My Digital Life where some nefarious user has managed to extract the .cab files from a Windows Server 2012 Standard installation required to allow DISM to install the feature. While the post is targeted at Windows 8 64-bit users to use dedup on their desktop machines, the process works equally well for Windows Server 2012 Essentials, if not better as you can also use the GUI to drive the configuration.

I don’t want to be the one in breach of copyright infringement or breach of terms of service with Microsoft, so I’m not going to link to the .7z file provided on My Digital Life, so download it from them, sorry.

Download the file and extract it to a location on the server. Once extracted, open an elevated command prompt, change the directory context of the prompt to your extracted .7z folder and enter the following command:
dism /Online /Add-Package /PackagePath:Microsoft-Windows-VdsInterop-Package~31bf3856ad364e35~amd64~~6.2.9200.16384.cab /PackagePath:Microsoft-Windows-VdsInterop-Package~31bf3856ad364e35~amd64~en-US~6.2.9200.16384.cab /packagepath:Microsoft-Windows-FileServer-Package~31bf3856ad364e35~amd64~~6.2.9200.16384.cab /PackagePath:Microsoft-Windows-FileServer-Package~31bf3856ad364e35~amd64~en-US~6.2.9200.16384.cab /packagepath:Microsoft-Windows-Dedup-Package~31bf3856ad364e35~amd64~~6.2.9200.16384.cab /PackagePath:Microsoft-Windows-Dedup-Package~31bf3856ad364e35~amd64~en-US~6.2.9200.16384.cab

If DISM fails or gives you any errors, then the most likely cause is that you didn’t use an elevated command prompt. The next likely cause is that you aren’t in the correct working directory so check that too.

Once all of the packages are imported okay, enter the second command:
dism /Online /Enable-Feature /FeatureName:Dedup-Core /All

No restart is required for the import of the packages or the enabling of the feature, so everything can be done online.

Once the feature is enabled, head over to Server Manager to get things started. Server Manager isn’t pinned to the server Start Screen by default, so from the Start Screen type Server Manager and it will appear in the in-line search results.

From Server Manager, select File and Storage Services from the left pane, and then select Volumes from the sub-options.

As you will see in the screenshot, I’ve already enabled dedup on the volume on this test Windows Server 2012 Essentials VM of mine and I’ve saved space  by virtue of the fact that I’ve created two data folders with identical data in each folder.

For you to configure your volumes, right click the volume you want to setup and select the Configure Data Deduplication option. On the options screen, first, tick the box to enable the feature. Once selected, you have options for age of files to include in Deduplication and types of file to exclude. For my usage at home, I am setting the age to 0 days which includes all files regardless of age, and I am choosing to not exclude any file types as I want maximum savings.

The final step is at the bottom of the dialog, Set Deduplication Schedule. This allows you to configure when optimization tasks occur and whether to use background optimization during idle periods of disk access. I chose to enable both of these and I have left the default time of 0145hrs in place.

Once you click OK and then OK again on the initial dialog, you have just enabled dedup on that volume. Repeat the process for any volumes you are interested in and job done for you. After this, the server has the hard task of calculating all the savings and the process of actually creating the metadata links to physical blocks on the disk and marking the space occupied by duplicate blocks on the disk as free space. This process is very CPU and memory heavy and depending on the size of your dataset can and will take a long time to run.

I am just about to kick off a manual task on my live Essentials server at home, so once the results are in, I will be posting here to report my savings and also the time taken, but I’m not expecting this to come in anytime within the next day or so.

 

The Problem with Storage Spaces

Richard Green on

As you may well have gathered from a number of my previous posts about Windows Server 2012 and Storage Pools, I was intending on using them for my home server rebuild, and I am indeed using them, however I have neglected to post anything showing the new server (although I will change that shortly).

I ran into a problem with Storage Pools today which I think quite frankly blows. I got myself a new Western Digital 3TB Red drive to try out. The plan is to replace all of my existing six 2TB Western Digital Green drives with these for a number of reasons including greater bang for buck on power consumption, increased IOPS, cooler running temperature and improved reliability.

Not wanting to keep a mixture of Green and Red drives for very long, I proceeded to remove one of the drives from the pool to replace with a Red drive. The Storage Pool refused to remove it as a Simple non-redundant Storage Space was being hosted on this drive.

Problem 1:  Storage Spaces cannot be converted between Simple, Mirror or Parity. Once they are created, they are created. My only option for this was to create a new temporary Space marked as Mirror and copy the data from the Simple so that I could delete it. Once deleted, I tried a second attempt to remove the drive and I got an error that I needed to add another drive to the pool as there was insufficient capacity.

I’m sorry, what?

Problem 2: I have six 2TB drives in an uber-pool. I am currently less than half of it, so removing the drive should be no problem. I tried this a few more times and each time I got the same error that I would need to add more capacity to the pool before I would be able to remove the drive, which I know to be cobblers.

In the end, I just pulled the disk from the server and let the Storage Pool have a cry about the missing disk. From here, I marked the disk for removal to allow Windows to think that the disk was failed and that it was never coming back. This worked although is time consuming as it forces all Mirror and Parity virtual disks to enter a repairing state, copying blocks to remaining disks in the pool to keep up the protection level.

This brings me softly onto another point which is more of a beef.

Beef 1: One of the tricks of Windows Server 2012 was deduplication. Anyone familiar with Windows Server 2012 will know that Storage Pools and deduplication do work together, but in Essentials, deduplication is absent, missing, not there. The feature is completely missing from any of the Server Manager interfaces and from PowerShell, the command Get-Command -Module Dedup* returns nothing.

Why is it missing from Essentials? Essentials is the release of Windows Server 2012 targeted at SMB/SME and pro-home customers, the customers most likely to be storing a lot of data on a tight budget, so why strip out the feature that they will probably be highly interested in, in Windows Server 2012.

I really hope that Microsoft get enough complaints from customers of Essentials to release a Feature Update to re-add the support for deduplication.

With this done,

Storage Architecture for Windows Server 2012 Essentials

Richard Green on

Two of the best features in my eyes in Windows Server 2012 Essentials over Windows Home Server 2011 are both related to disk.

RAID Support
Windows Server 2012 Essentials is a grown-up Windows Server unlike Windows Home Server 2011 which in an aim to simplify the server in the home idea for consumers, removed the ability to use hardware RAID the operating system volume. This was a horrible thing for Microsoft to do in my opinion.

Storage Spaces
In a nod to Driver Extender from Windows Home Server (v1) Windows 6.2 kernels running on Windows 8 and Windows Server 2012 both support Storage Pools and Storage Spaces. This allows you to pool disks together to produce simple, mirrored or parity volumes from a single pool of disks. It’s like RAID on steroids because it means you only waste the chunks on disk that you want to for volumes that you want to protect, not all of them.

So taking these two ideals into consideration, what am I going to do?

Step 1 is to get the operating system off of the pair of 2TB disks I have, where there is a 60GB partition for the OS and a 1.8TB partition on one disk, and a 1.8TB partition on the second mirrored from the first disk using Windows Disk Management mirroring.

Step 2 is to maximize the utilization of the capacity of my six 2TB disks.

To achieve step 1, I am investing in a pair of SSD disks. For Windows Server 2012 Essentials to accept them they have to be over 160GB, so I am looking at the Intel 520 Series 240GB disks which are currently on Amazon reduced from £300 to £180. These will be connected to my SATA RAID controller in a RAID1 mirror and will be installed in a Lian Li 5.25″ to dual 2.5″ adapter, allowing me to utilize one of the three 5.25″ bays in my case which I will not ever use otherwise, opening up two slots for 3.5″ high capacity disks for future expansion. Needless to say, a pair of Intel 520 Series 240GB disks will give the operating system volume unbelievable IOPS and will allow the server to boot, reboot and access the OS extremely quickly. I’m also going to leave it as one super-sized 240GB partition so that I never have to worry about Windows Updates or software I install on the server causing me to need to think about repartitioning in the future.

To achieve step 2, it’s simple. Connect the now completely free to breath six 2TB disks to any of the on-board or two remaining SATA RAID controller ports, and configure them in Windows Server 2012 Essentials as a single, six disk Storage Pool and carve my volumes out of this 12TB raw disk pool using the protection levels I see fit for my needs.

Thanks to the ability to over provisioning (or thin provisioning as Microsoft incorrectly refer to it in my opinion) on Storage Spaces, I can create spaces larger than my current capacity and add disk or replace existing 2TB disk with 3TB or 4TB disk as available to extend the live capacity.

Over time, as I require more disk there will be one ‘problem’ in that I will have depleted all of my SATA ports. Luckily, my SATA RAID controller supports Port Multipliers, and a cheap and potentially nasty Syba 5 to 1 SATA Port Multiplier for about £45 means I can extend my capability to an extra four ports which at that point reaches the capacity of the case chassis. Power also isn’t an issue as my Corsair AX750 power supply was selected at the time specifically because of it’s amazing ability to run at peak power efficiency at extremely low consumption levels and also to support up to 12 SATA disks with its modular cabling design.

So there we have it…my design for Windows Server 2012 Essentials Storage Architecture. It’s by no means conventional but then I don’t really think anything about my server build is, with it’s 4U rack mount configuration packing a build-out consuming less power than your average light fixture.

I only designed and stood up the Windows Home Server 2011 setup little over a year ago. I think we all secretly knew that Home Server as a product family was a dying breed and that Microsoft would either kill it off completely or encompass it into another product family sooner than later to drop the support overheads. Thankfully it happened sooner I feel: Yes, it means that I have to rebuild my setup not that long after it was actually first built, but thankful because it meant I haven’t invested too heavily in customisation or further expansion of my current setup leaving me playing the corner flag with a legacy product at work. Luckily now, with Windows Server 2012 Essentials being a core SKU in the Windows Server family, it will be three years until the next major release. Although a Windows Server 2012 R2 release may appear sometime in the middle of the three year release cadence for server operating systems, at least being on the RTM release for the same product should make that migration a hell of a lot easier.