feat: Epyc post

This commit is contained in:
Kyle Brown 2024-01-13 16:42:45 -08:00
parent 702154827a
commit 771a6b3bf6
5 changed files with 80 additions and 3 deletions

View file

@ -0,0 +1,77 @@
+++
title = "An Epyc Change"
date = "2024-01-13"
[taxonomies]
tags = ["linux"]
+++
# I got tired of managing systems
After messing with GlusterFS for a while, learning it's ups and downs, I've
determined that I'm moving back to a single physical node, for the most part. I
had a few goals in mind when looking at hardware.
- Enough CPU/RAM to replace everything I currently run
- PCIE expansion. Lots of lanes for everything I ram in there.
- IPMI! I don't want to have to go plugging in display/keyboard constantly.
This pretty much left me with 2 major system types to look at. Multi socket
Intel Xeon's, or AMD Epyc systems. With old Xeon systems being even more work
due to the absolute requirement of understanding [UMA and
NUMA](https://www.geeksforgeeks.org/difference-between-uniform-memory-access-uma-and-non-uniform-memory-access-numa/)
and more importantly going through the steps of setting it up, higher power
draw, and with that noise, I quickly ruled them out. I set my eyes on the [Epyc
7551p](https://www.techpowerup.com/cpu-specs/epyc-7551p.c1929) and the
[Supermicro
H11SSL-NC](https://www.supermicro.com/en/products/motherboard/H11SSL-nc)
motherboard to cover every single thing I wanted out of the system. At the cost
of about $700 USD to fully replace the core of my old system, I felt like this
was a reasonable price for what all I get.
## Specs
- [AMD Epyc 7551p 32 core (64 thread)](https://www.techpowerup.com/cpu-specs/epyc-7551p.c1929)
- [Supermicro H11SSL-NC](https://www.supermicro.com/en/products/motherboard/H11SSL-nc)
- 128GB Samsung 2133 Registered ECC
- 2x256GB [Inland Professional SSD](https://inlandelectronics.com/product/inland-professional-240gb-sata-ssd/)
- 3x10TB Western Digital drives (white label pro red)
- 2x1TB [Crucial MX500](https://www.crucial.com/products/ssd/crucial-mx500-ssd)
- 2x2TB [Intel P3600 PCIE SSD](https://www.intel.com/content/dam/www/public/us/en/documents/product-briefs/intel-ssd-dc-family-for-pcie-brief.pdf)
- [Fractal Define R5](https://www.fractal-design.com/products/cases/define/define-r5/black/)
- [HP NC550SFP](https://www.hpe.com/psnow/doc/c04111686?jumpid=in_lit-psnow-red)
- LSI SAS HBA
![mainboard.jpg](mainboard.jpg)
## Build issues and complications
Over all, this was a fairly simple build in terms of hardware. The Motherboard
is *mostly* ATX standard with one exception (photo later). One oversight I had
was not having VGA anything to hook up to the system when it showed up. The
onboard video is forced to be the only enabled display by default, and I should
have seen that coming. IPMI is also disabled by default, so wasn't able to even
remote into it until I got a display out.
Turns out it's *mostly* ATX. Whoops, need to remove a standoff.
![standoff.jpg](standoff.jpg)
## The plan!
The current goal is to get it running on Proxmox to allow for easy separation of
duties, and isolation of public/private hosted services. I'm planning on running
something to manage ZFS and act as a SAN for the network, probably [TrueNAS
Scale](https://www.truenas.com/truenas-scale/) as I have recommended that many
times, and want to see what it's like today, despite knowing well how to manage
a fully headless system. I'm pulling out a second gen Ryzen 8 core system of the
network, so I'll also be creating a virtual machine to directly replicate it's
place to make transitioning easy. After that, it's up to whatever I want. I'll
be hosting a semi public [Nix Hydra](https://github.com/NixOS/hydra) instance
for friends since I have too much CPU grunt going to waste. I'll also be able to
turn back on transcoding on my [Jellyfin](https://jellyfin.org/) server as I
know this should easily be able to transcode 4 or more streams while leaving
plenty left for the rest of the services. After that, who knows.
What it looks like thus far.
![setup.jpg](setup.jpg)

Binary file not shown.

After

Width:  |  Height:  |  Size: 3.1 MiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 445 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 268 KiB

6
flake.lock generated
View file

@ -20,11 +20,11 @@
},
"nixpkgs": {
"locked": {
"lastModified": 1702272962,
"narHash": "sha256-D+zHwkwPc6oYQ4G3A1HuadopqRwUY/JkMwHz1YF7j4Q=",
"lastModified": 1704842529,
"narHash": "sha256-OTeQA+F8d/Evad33JMfuXC89VMetQbsU4qcaePchGr4=",
"owner": "nixos",
"repo": "nixpkgs",
"rev": "e97b3e4186bcadf0ef1b6be22b8558eab1cdeb5d",
"rev": "eabe8d3eface69f5bb16c18f8662a702f50c20d5",
"type": "github"
},
"original": {