One Server: Researching the hardware
Using my list of requirements, I set out to find the hardware for my new server.Â I was building this from scratch so at minimum my purchase list needed to include
- hard drive storage
- server case
- motherboard, RAM & CPU
Hard drive storage
I decided to focus first on the requirements for the fileserver side of the project.Â Recall that I was planning for 16TB of storage space.Â At the time, the largest consumer hard drives were 2 TB.Â I also wanted to be able to support multiple drive failures and be able to replace the drives without shutting down the system.Â That meant I needed at least 10 hard drives.
When researching the hardware for this server I came across a good blog post from Adaptec about real life RAID reliability.Â That article compared the reliability ofÂ RAID-5 and RAID-6 arrays and showed that a RAID-6 array should last 172 times longer than a RAID-5 array.Â Reliability was important to me on this project, so I decided to go with RAID-6.Â The Adaptec article only considered enterprise grade drives. I planned to build this server with consumer grade drives.Â Therefore, as a precaution, I chose to add two extra drives as hot spares.
A RAID-6 array with 10 drives was likely to run slow.Â So I did some more searching and came across RAID-60.Â RAID-60 combines the redundancy of RAID-6 with the speed of striping found in RAID-0.Â However, to get 16 TB with RAID-60, and have two hot spares, I now needed 14 hard drives.Â Six drives for each of the two RAID-6 arrays and two hot spares.
I wanted to make sure the fileserver would run quickly so that I could stream video from it while MythTV was also recording new programs and all the virtual machines were also running.Â I thought running everything off one set of storage drives might be too much, so I decided to split the VM storage from the NAS storage.Â That meant adding additional drives.Â I had four 1TB drives from my previous NAS, so I decided to use them for storing the VM images.
That put the total number of hard drives needed at 18.Â This was shaping up to be quite a storage server!Â The next task was determining how to fit that many hard drives into a computer case.
I wanted the server to be able to stay running while I replaced a failed drive, so I needed a case that accommodated hot swappable drive bays.Â I considered putting the drives into some six-bay external drive enclosures, but decided that would get too expensive and end up using more power then was needed.Â Plus, I could just see the cables getting disconnected between the external enclosures and the main CPU.
No tower-style case that I could find would hold that many drives, so I looked for rack mountable cases.Â Â To fit 18 drives, a 4U rack mount case was needed.
Motherboard, RAM & CPU
I wanted to be able to expand this system in the future, so when choosing the motherboard I focused on server boards that supported dual CPUs.Â My plan was to put the system together with one CPU, and if needed, add another CPU later.Â I also needed to find a motherboard with multiple network interfaces, and plenty of PCI-express slots for adding RAID cards.Â Since reliability was important to me, I focused only on motherboards that supported ECC RAM.Â Form factor wasn’t a big issue for this system as it was being placed into a rack mount case with plenty of space.
For the CPU, I needed a processor that supported VT-d.Â VT-d processors support mapping cards plugged into PCIe slots directly into virtual machines.Â My goal was to create a virtual machine for the fileserver and map the RAID card directly into that VM.
Another goal of mine was to make the new server easy to administer.Â I didn’t want to have to find a spare keyboard, mouse, and monitor and plug them all in when there was trouble.Â The solution, IPMI.Â A motherboard with IPMI would allow me to remotely control the keyboard, mouse, video and even attach a remote DVD-ROM to perform an OS install.Â It is basically a built-in KVM over IP.Â I can even remotely reset the computer using IPMI.
I ended up purchasing the following components for this system
- 14 x 2 TB hard drives (5 x Hitachi HDS72202, 3 x SAMSUNG HD203WI, 6 x WDC WD20EADS)
- Norco RPC-4220 4U rackmount case with 20 hot swappable drive bays
- Supermicro X8DTi-F motherboard with 3 PCIe 8x slots and IPMI
- 24Gb ECC Registered DDR3 1066 RAM
- Intel Xeon E5506 Nehalem-EP 2.14 GHz processor