Basic Homelab Setup

As an IT Professional, I have a home lab to experiment and learn things on. It is part experimentation, and part functional requirements.

I’ll make a few posts to describe what I have, and how I have it set up.

Picture

Hardware

Let’s start off with the basic hardware.

Cable modem

ISP Provided DOCSIS 3.0 modem with a Fixed IP, 240 / 30 Mbps, and no BW caps.

Switch

A TPLink Smart Switch T1700G-28TQ

24 10/100/1000Mbps RJ45 Ports
4 fixed 10G SFP+ Slots
Fanless

Nothing special, has a bunch of VLANs configured on it.

Router

A UBNT EdgeRouter-4

3 x 1000BASE-T ports
1 x SFP port
Up to 3.4 million pps
Up to 4 Gbps linerate

Has a IPSec/GRE tunnel to the VyOS VPS instance running BGP. Routes between the various subnets,

$ show interfaces
Codes: S - State, L - Link, u - Up, D - Down, A - Admin Down
Interface    IP Address                        S/L  Description
---------    ----------                        ---  -----------
eth0         203.0.113.6/18                    u/u  WAN
             2001:DB8:1:161:ede7:eac3:46f6:6969/128
eth1         192.168.13.1/24                   u/u  LAN
             2001:DB8:3920:400:feec:daff:fe44:14f/64
eth1.200     192.168.0.201/24                  u/u  Restore Ranges
             192.168.1.201/24
eth1.300     192.168.9.1/24                    u/u  Guest
eth1.700     192.168.7.1/24                    u/u  Internet of Things
eth2         -                                 A/D  Spare
eth3         -                                 A/D  Spare
lo           127.0.0.1/8                       u/u
             ::1/128
tun0         192.168.213.13/30                 u/u  GRE to VyOS VPS
             2001:DB8:cbb9::13:2/126

$ show ip bgp summary
BGP router identifier 192.168.13.1, local AS number 4200000013
BGP table version is 2
1 BGP AS-PATH entries
0 BGP community entries

Neighbor                 V   AS   MsgRcv    MsgSen TblVer   InQ   OutQ    Up/Down   State/PfxRcd
192.168.213.14           4 4200000000 79253      79265       2      0      0  02w0d21h               0
2001:DB8:cbb9::13:1      4 4200000000 79258      79267       2      0      0  02w0d21h               0

Total number of neighbors 2

Total number of Established sessions 2

Wifi

An UAP-AC-PRO

802.11ac
3x3 11AC MIMO
5 GHz and 2.4 GHz
Band Steering
Airtime Fairness
PoE

The main LAN subnet is on 5Ghz, and the Guest and IoT subnets are on 2.4Ghz (keep those good waves for meself, eh?)

Storage

There are 2 NAS setups.

DIY

A DIY setup consisting of:

C2750D4I Board
32GiB ECC Ram
6 x 3T SATA Disks
- in RAIDZ2
Mellanox MT26448 SFP+ network card
in a Node 304 Case

Runs Plex, exports some iSCSI LUNs, full backup copy of the Synology DS218+ and is for general playing around when I need storage.

# zpool list
NAME   SIZE  ALLOC   FREE  EXPANDSZ   FRAG    CAP  DEDUP  HEALTH  ALTROOT
tank  21.8T  20.8T   998G         -    39%    95%  1.00x  ONLINE  -
# zpool status tank
  pool: tank
 state: ONLINE
  scan: scrub in progress since Sat Jun 15 18:07:12 2019
    393M scanned out of 20.8T at 4.05M/s, (scan is slow, no estimated time)
    0 repaired, 0.00% done
config:

    NAME        STATE     READ WRITE CKSUM
    tank        ONLINE       0     0     0
      raidz2-0  ONLINE       0     0     0
        sdb     ONLINE       0     0     0
        sdc     ONLINE       0     0     0
        sdf     ONLINE       0     0     0
        sde     ONLINE       0     0     0
        sdg     ONLINE       0     0     0
        sdd     ONLINE       0     0     0

errors: No known data errors

Synology

A Synology DS218+

64bit Intel Celeron J3355
2G Ram
2 x 3T disks in Raid 1

This one is for important stuff like photos, and business documents.

ESXi Server

An Intel NUC NUC6i5SYK:

32GiB Ram
512GB Samsung NVMe Flash Storage

Runs VMWare ESXi 6.5

UPS

APC Back-UPS 1400U

All the above is connected to the UPS, with DS218+ connected over USB as well.

Network

Every subnet is on its own VLAN.

VLAN100: LAN
- General LAN range. Most things are on this.
VLAN200: Restore ranges
- A special NATed range. Useful for needing to access new / old devices that expect to be in those ranges.
- Has subnets 192.168.0.123/24 and 192.168.1.123/24 on it
VLAN300: Guest
- Guest access on the AP. So I can give wifi to people I don’t really want on my WAN :P
VLAN700: IoT
- General IoT devices, TV, AVR, etc. Stuff I don’t trust but need some sort of network anyway. Heavily filtered on the firewall.
VLAN2000: WAN
- This is for my public IPs. The cable modem and the router are on this VLAN.

Network Diagram

Software

Rancher / docker

A VM on the ESXi, running RancherOS with Rancher as a frontend.

Soon to be upgraded to K3OS.

Traefik

The ingres point is a Traefik instance, taking its input from Rancher. It is set up to do automatic TLS termination with Let’s Encrypt.

Unifi

Controller software for the UAP-AC-PRO

IRC

An instance of the ZNC IRC Bouncer.
An instance of The Lounge web IRC client, connected to the ZNC bouncer.

Monitoring

Prometheus metrics database
InfluxDB metrics database
Telegraf metrics collector
Grafana dashboard

Home Automation

I have a Philips Hue hub and lights, and a Z-Wave system with power plugs and various sensors. A Mosquitto MQTT server is used as a message bus.

Home Assistant is used for connecting to various protocols, and Node Red for handling the flow logic.

TOC