Tau-Volantis Storage Server – Upgrades Part 1

So I figured it was time to get a hardware update sorted for my network’s core storage server, which I have posted about before. The way I had the drives anchored to steel rails really doesn’t make moving or replacing disks easy, so a proper case needed to be sourced.

ServerCaseUK stocked 16-bay 4U chassis units, so one of these was ordered. These have 4 internal backplanes, with SFF-8087 Mini-SAS connections, so hooking into my existing 16-channel HBA card would be simple. In the current setup, the multi-lane cables are routed out via SFF-8088 connectors to the drive array, so this will tidy things up considerably.

The main data links are via these SFF-8087 connectors, each carrying 4 lanes of SATA.

Power is provided by 4x Molex connections, via SATA power adaptors (the good kind, which don’t create fire). There’s a 5th Molex hidden down the size of the last fan, which powers all 3 120mm fans.

The disks are kept cool by 3x 120mm hot-swap fans on the dividing wall. These don’t create much noise, and are always at full speed.

Here’s the back of the case after transplanting the motherboard & HBA from the old chassis. There’s a new 750W EVGA modular power supply, since I’ll be expanding the disk array as well. The boot SSD is currently sat on the bottom of the case since I don’t have a data cable long enough to mount it in the proper place as yet.

Here’s the fan controller, which takes care of the dual high speed Delta fans on the back wall of the chassis. This has a pair of temperature sensors – one on the HBA card’s heatsink, and the other on the fan wall monitoring the exhaust air temp of the drive array, to control the speed of the two fans. Temperatures are kept at around 30°C at all times.

Since the HBA card’s fan failed a while back, it’s had a couple of fans attached. The centrifugal one here works a little better than a massive 80mm axial fan, and is a little quieter. This is always run at full speed from a spare motherboard header. The temperature sensor feeding the fan controller can be seen here bonded to the heatsink. The 4 SFF-8087 cables are going off to the disk backplanes.

As mentioned before, there are a pair of 80mm Delta high-speed fans on the back wall of the case, to provide some extra cooling air flow just in case overheating manages to set in. These are usually spooled down to low RPM to keep them quiet.

Since space was getting a little tight, and having some slots spare on the HBA, I decided to add some more disks to bring the active members up to 12, from 9 – increasing available disk space from 28TB to 40TB.

/dev/md0:
        Version : 1.2
  Creation Time : Wed Mar 11 16:01:01 2015
     Raid Level : raid6
     Array Size : 39068874880 (37258.98 GiB 40006.53 GB)
  Used Dev Size : 3906887488 (3725.90 GiB 4000.65 GB)
   Raid Devices : 12
  Total Devices : 13
    Persistence : Superblock is persistent

  Intent Bitmap : Internal

    Update Time : Mon Nov 18 14:13:35 2019
          State : active 
 Active Devices : 12
Working Devices : 13
 Failed Devices : 0
  Spare Devices : 1

         Layout : left-symmetric
     Chunk Size : 64K

           Name : Main-PC:0
           UUID : 266632b8:2a8a3dd3:33ce0366:0b35fad9
         Events : 1653174

    Number   Major   Minor   RaidDevice State
      16       8      144        0      active sync   /dev/sdj
      11       8      160        1      active sync   /dev/sdk
       9       8      176        2      active sync   /dev/sdl
      10       8      208        3      active sync   /dev/sdn
      15       8      192        4      active sync   /dev/sdm
       5       8      112        5      active sync   /dev/sdh
       6       8       80        6      active sync   /dev/sdf
      13       8       96        7      active sync   /dev/sdg
       8       8       64        8      active sync   /dev/sde
      19       8        0        9      active sync   /dev/sda
      18       8       48       10      active sync   /dev/sdd
      17       8      128       11      active sync   /dev/sdi

      12       8       16        -      spare   /dev/sdb

The space expansion improves things there, I will be adding a couple more spare disks to bring the number up to the full 16, just in case of any failures.

There are still a couple of issues with this setup:

• The motherboard & CPU are ancient. Currently an Intel Core 2 Quad, running 8GB of RAM, limits data throughput, and critically, the speed of mdadm data checks & rebuilds. The Core 2 Quad also runs at roughly the same temperature as the Sun’s core when under high load.
• The SATA HBA is running 4 controllers on an expander, through a PCIe x4 link, which is a little slow due to congestion on the expander itself. RAID6 does have some write-speed penalties though.

These are issues I will address shortly, with a replacement motherboard on the way!