Automotive Calulators
Front Yard
Data Center North
Solar backup power
Water cooled PC

Data Center Dublin North - Open Compute Windmill deployment


Deploy a small two through four node cluster of machines in a "hyper converged" infastructure method. The cluster must support virtual machines and should support containterized applications if possible.

Deploy an OpenStack "Compute Start Kit" four node group of machines. Primary services will be virtual machines images(KVM), container images (Podman or Docker).

OpenStack Docs

Technical Implementations


Consider taking a look at oVirt

  1. GlustFS config for storage.
  2. Figure out oVirt.

Node disposition

Node 1192.168.0.201/24192.168.100.201/24
Node 2192.168.0.202/24192.168.100.202/24
Node 3192.168.0.203/24192.168.100.203/24


The first attempts at running StarlingX failed. The hardware requirements of 64GB of RAM and two storage devices was not meet. I may revisit this configurtion by canabalizing RAM from one of the other nodes and attempting to use an NVMe drive as a PCIe device to meat the minimum requirements.

Kubernetes cluster

Disable swap comment out swap line in /etc/fstab Disable IPv6 https://linuxconfig.org/how-to-disable-ipv6-address-on-ubuntu-20-04-lts-focal-fossa update /etc/default/grub GRUB_CMDLINE_LINUX_DEFAULT="ipv6.disable=1" then run update-grub and reboot Network configuration bond0 mode: balance-alb

VMWare to NFS datastore using kernel NFS v4

The goal for this testing is to have an ESXi 6.7 host connected to a NFS server via NFS 4.1 protocol. The storage will be backed via a GlusterFS clustered volume.

Using virt-mgr I was able to get ESXi 6.7 installed as a guest machine.

This ran into quite a few issues with NFS and ESX. ESX does not support multipath NFS. ESX was completely unable to mount NFS volumes presented by NFSGanesha. ESX did connect to Kernel NFS but was "flakey".

Ubuntu Base OS configuration

  1. Ubuntu 20.04
  2. Configure IP addresses for service and backend networks.
  3. apt install ssh
  4. apt install firewalld
  5. apt install smartmontools
  6. apt install chrony
  7. apt install xfsprogs - and configure XFS volumes on each drive.
  8. sudo apt install glusterfs-server
  9. apt install software-properties-common Then add the community GlusterFS PPA: add-apt-repository ppa:gluster/glusterfs-7 apt update Finally, install the packages: apt install glusterfs-server

Base CentOS Stream configuraiton

  1. CentOS 8 minimal install
  2. dnf install dnf-automatic
  3. systemctl enable dnf-automatic-timer
  4. edit /etc/dnf/automatic.conf change "apply_updates = yes"

MEI fix for Windmill 2.0 server

  1. (modprobe -r mei echo "blacklist mei" >> /etc/modprobe.d/local-blacklist.conf echo "install mei /bin/false" >> /etc/modprobe.d/local-blacklist.conf)
  2. echo "blacklist mei" >> /etc/modprobe.d/local-blacklist.conf
  3. echo "install mei /bin/false" >> /etc/modprobe.d/local-blacklist.conf

Chrony NTP configuration

  1. Create directory to use for NFS share

Network configuration

  1. Use chronyd to synchronize to know time source
  2. To be determined, firewall currently disabled. Recomendations are to have a dedicated network interface for gluster node communication and not firewall this interface on the network.

Drive configuration

  1. mkfs.xfs /dev/vdb

GlusterFS configuration

  1. dnf install centos-release-gluster
  2. dnf install glusterfs-server
  3. mkdir gv0
  4. gluster volume create gv0 replica 2
  5. gluster volume start gv0

Kernel NFS configuration

  1. dnf install nfs-utils
  2. Edit /etc/exports /<directory name>,all_squash)

NFS-Ganesha installation methodology

Having trouble with nfs-ganesha30 package on CentOS Stream. Trying nfs-ganesha28

  1. dnf install dnf-automatic
  2. edit /etc/dnf/automatic.conf change "apply_updates = yes"
  3. dnf install centos-release-nfs-ganesha30
  4. yum -y install nfs-ganesha nfs-ganesha-yum -y install nfs-ganesha nfs-ganesha-vfs
  5. dnf --enablerepo=centos-nfs-ganesha3
  6. sed -i -e "s/enabled=1/enabled=0/g" /etc/yum.repos.d/CentOS-NFS-Ganesha-3.repo
  7. dnf -y install centos-release-nfs-ganesha30

Argh... CentOS stream's package for nfs-ganesha30 does not include vfs. Switched to Ubuntu 20.04, please note much newer kernel

  1. Install Ubuntu server 20.04
  2. sudo apt install nfs-ganesha nfs-ganesha-vfs
  3. Configure /etc/ganesha/ganesha.conf
EXPORT { Export_Id = 5; Path = /nfs; Pseudo = /nfs; Protocols = 4; Access_Type = RW; #Squash = root_squash; Sectype = none,sys; FSAL { Name = vfs; } }

Hardware deployment

OpenCompute Windmill server
DescriptionQuantityPriceExt. Price
240v outlet2
Cover plate2
Double gang box2
Single gang box2
Windmill R2.0 Server4
Winbond 25q128bvfg0
Intel 2680 CPU2
Intel 2665 CPU2
Micron 8GB DDR316
OEM (Dell) AMD 64501

Network configuration via ONOS.

  1. Temperature monitoring via ipmi?
  2. BMC does not provide console access.
  3. DCIM, can I get it working?
  4. Reboot issued through OS does not restart machine without a power cycle. Is it possible to write CoreBoot to the firmware instead?

Problem list