Hardware
Four processors, 192 GB of memory and twenty-four drive bays, bought retired and rebuilt. Every part below is a link on the drawing.
The machine
The machine, drawn from its manual and its own inventory rather than photographed. Every part is a link. Choose one on the drawing, or from the list of parts beneath it, and what it is and does here opens over the page.
One panel · power, identify, console
Control panel
The power button, a non-maskable-interrupt button for a wedged operating system, the identify button that lights the machine in a rack, a small status screen, two USB ports and a video port. Everything on it can also be done remotely through the out-of-band controller, which is how this machine is normally handled.
connected to
One bay · empty
Optical drive bay
The chassis has a slot for a DVD drive. Nothing is installed; the machine boots from its own disks and was installed from a USB stick.
Eight bays · empty
PCIe flash bays
Two groups of four bays wired for PCIe flash drives, the fastest storage the chassis can take. None are fitted: the estate does not need them, and the room to grow is part of why the machine was chosen.
connected to
Sixteen bays · four in use
SAS bays
Sixteen bays for 2.5-inch drives on the SAS backplane. Four hold the estate's solid-state drives; twelve are free; with the flash bays, twenty sit empty across the chassis. Every drive can be pulled and replaced while the machine runs.
connected to
Four drives · two mirrored pairs
Solid-state drives
Two 3.6 TB consumer drives and two 894 GB enterprise drives, each pair mirrored by ZFS: the small pair holds the system, the large pair holds everything else. ZFS checksums every block, so a corrupted read is caught and repaired from the other half of the mirror.
connected to
Four supplies · 1100 W, redundant
Power supplies
Four hot-swappable supplies in two pairs. The machine runs on any one of them; all four are cabled, so a supply failing changes nothing but a light. The whole chassis draws about 260 W at idle.
connected to
Eight slots · all free
Expansion slots
Eight PCIe slots between the supplies, two of them full length. The network and storage controllers are on their own risers, so every slot is free: a graphics card was considered and set aside, and the room stays.
connected to
One controller · iDRAC, always on
Out-of-band controller port
The controller has its own processor, network port and firmware, and runs whether the server is on or off. It gives power control, a remote console and the hardware inventory. Its port sits on the household network and answers only a short list of addresses there; nothing reaches it from the internet.
connected to
Four ports · two 10 GbE, two 1 GbE
Network ports
Four ports on the network daughter card: two 10-gigabit and two 1-gigabit. One carries the household network and, over it, the private mesh; the 10-gigabit pair waits for a switch that could use it.
connected to
Four ports · for a crash cart
Serial, video and USB
A serial port, a video port and two USB ports for a keyboard and screen wheeled up to the machine. In practice the remote console replaces all of them.
connected to
One board · every bay to the controller
Drive backplane
Behind the front bays, the board that carries every drive to the storage controller and lets a drive be swapped live.
connected to
Six fans · redundant
Cooling fans
Six fans in a row behind the drives pull air from the front through the processors and out the back. They are redundant: five keep the machine within limits. When the out-of-band controller restarts they run flat out for a few minutes, which is loud and not a fault.
connected to
Four sockets · 72 cores, 144 threads
Processors
Four Xeon E7 processors of eighteen cores each under four large heat sinks. Far more than the estate uses: the load sits near zero, and the containers share a machine built for a database.
connected to
Eight risers · 192 GB fitted
Memory risers
Eight vertical risers of twelve slots each. Four risers carry three 16 GB modules apiece, 192 GB in all, of which the containers use a small fraction; the other four risers are empty, and the ceiling is over a terabyte.
connected to
One card · pass-through, not RAID
Storage controller
The hardware RAID controller, deliberately not doing RAID: in pass-through mode it hands each disk to the operating system whole, so ZFS can checksum and repair them itself. A pool built this way can be read by any machine; an array built by the controller could be read only by its own kind.
connected to
- Chassis
- Dell PowerEdge R930 — 4U, quad-socket, refurbished
- Processors
- 4 × Intel Xeon E7-8890 v3 — 72 cores, 144 threads
- Memory
- 192 GB DDR4 ECC — 188 GB usable
- Storage
- Enterprise and consumer SSD, fully mirrored — 20 bays free
- Network
- Dual 10 GbE and dual gigabit
- Management
- Dedicated out-of-band controller — power and console independent of the OS
- Power
- Redundant supplies, about 260 W at idle