hot-add CPU and memory to Ubuntu guest in VMware

por | 22 enero, 2020

Virtualization has made our lives easier. We can spin up new resources for applications, dynamically add more disk space, migrate them, etc. In vSphere, we can dynamically add additional memory and vCPU to a guest VM without any downtime (assuming its been enabled prior to powering on).

Now in Ubuntu 12.04, I had two production virtual machines that I needed to increase resources, but could not afford downtime (of course). Being the good sysadmin that I am, I spun up a test VM to ensure there is no downtime and last-minute gotchas. To my surprise, I was able to dynamically add more resources and vSphere recognized it, but the operating system did not. I stumbled across this Ubuntu forum and found the answer I was looking for: https://askubuntu.com/questions/764620/how-do-you-hotplug-enable-new-cpu-and-ram-in-a-virtual-machine

TL;DR: You need to “turn on” the resources. Change some 0s to 1s to bring them online. Once you add more resources, copy this script over to your guest VM and fire away.

I initially created my test VM with 4GB of RAM and dynamically added 8GB more (total of 12GB). Works like a charm

root@ubuntu-template:~# cat /proc/meminfo | more
MemTotal: 4040264 kB
root@ubuntu-template:~# ./hotadd.sh
Found cpu: "/sys/devices/system/cpu/cpu0" ...
 cpu0 already configured prior to hot-add
Found cpu: "/sys/devices/system/cpu/cpu1" ...
 cpu1 already online
Found ram: /sys/devices/system/memory/memory0/state:online ...
Already online
Found ram: /sys/devices/system/memory/memory100/state:offline ...
Bringing online
Found ram: /sys/devices/system/memory/memory101/state:offline ...
Bringing online
Found ram: /sys/devices/system/memory/memory102/state:offline ...
Bringing online
Found ram: /sys/devices/system/memory/memory103/state:offline ...
Bringing online
Found ram: /sys/devices/system/memory/memory10/state:online ...
Already online
Found ram: /sys/devices/system/memory/memory11/state:online ...
Already online
Found ram: /sys/devices/system/memory/memory12/state:online ...
Already online
Found ram: /sys/devices/system/memory/memory13/state:online ...
Already online
Found ram: /sys/devices/system/memory/memory14/state:online ...
Already online
Found ram: /sys/devices/system/memory/memory15/state:online ...
Already online
Found ram: /sys/devices/system/memory/memory16/state:online ...
Already online
Found ram: /sys/devices/system/memory/memory17/state:online ...
Already online
Found ram: /sys/devices/system/memory/memory18/state:online ...
Already online
Found ram: /sys/devices/system/memory/memory19/state:online ...
Already online
Found ram: /sys/devices/system/memory/memory1/state:online ...
Already online
Found ram: /sys/devices/system/memory/memory20/state:online ...
Already online
Found ram: /sys/devices/system/memory/memory21/state:online ...
Already online
Found ram: /sys/devices/system/memory/memory22/state:online ...
Already online
Found ram: /sys/devices/system/memory/memory23/state:online ...
Already online
Found ram: /sys/devices/system/memory/memory2/state:online ...
Already online
Found ram: /sys/devices/system/memory/memory32/state:online ...
Already online
Found ram: /sys/devices/system/memory/memory33/state:online ...
Already online
Found ram: /sys/devices/system/memory/memory34/state:online ...
Already online
Found ram: /sys/devices/system/memory/memory35/state:online ...
Already online
Found ram: /sys/devices/system/memory/memory36/state:online ...
Already online
Found ram: /sys/devices/system/memory/memory37/state:online ...
Already online
Found ram: /sys/devices/system/memory/memory38/state:online ...
Already online
Found ram: /sys/devices/system/memory/memory39/state:online ...
Already online
Found ram: /sys/devices/system/memory/memory3/state:online ...
Already online
Found ram: /sys/devices/system/memory/memory40/state:offline ...
Bringing online
Found ram: /sys/devices/system/memory/memory41/state:offline ...
Bringing online
Found ram: /sys/devices/system/memory/memory42/state:offline ...
Bringing online
Found ram: /sys/devices/system/memory/memory43/state:offline ...
Bringing online
Found ram: /sys/devices/system/memory/memory44/state:offline ...
Bringing online
Found ram: /sys/devices/system/memory/memory45/state:offline ...
Bringing online
Found ram: /sys/devices/system/memory/memory46/state:offline ...
Bringing online
Found ram: /sys/devices/system/memory/memory47/state:offline ...
Bringing online
Found ram: /sys/devices/system/memory/memory48/state:offline ...
Bringing online
Found ram: /sys/devices/system/memory/memory49/state:offline ...
Bringing online
Found ram: /sys/devices/system/memory/memory4/state:online ...
Already online
Found ram: /sys/devices/system/memory/memory50/state:offline ...
Bringing online
Found ram: /sys/devices/system/memory/memory51/state:offline ...
Bringing online
Found ram: /sys/devices/system/memory/memory52/state:offline ...
Bringing online
Found ram: /sys/devices/system/memory/memory53/state:offline ...
Bringing online
Found ram: /sys/devices/system/memory/memory54/state:offline ...
Bringing online
Found ram: /sys/devices/system/memory/memory55/state:offline ...
Bringing online
Found ram: /sys/devices/system/memory/memory56/state:offline ...
Bringing online
Found ram: /sys/devices/system/memory/memory57/state:offline ...
Bringing online
Found ram: /sys/devices/system/memory/memory58/state:offline ...
Bringing online
Found ram: /sys/devices/system/memory/memory59/state:offline ...
Bringing online
Found ram: /sys/devices/system/memory/memory5/state:online ...
Already online
Found ram: /sys/devices/system/memory/memory60/state:offline ...
Bringing online
Found ram: /sys/devices/system/memory/memory61/state:offline ...
Bringing online
Found ram: /sys/devices/system/memory/memory62/state:offline ...
Bringing online
Found ram: /sys/devices/system/memory/memory63/state:offline ...
Bringing online
Found ram: /sys/devices/system/memory/memory64/state:offline ...
Bringing online
Found ram: /sys/devices/system/memory/memory65/state:offline ...
Bringing online
Found ram: /sys/devices/system/memory/memory66/state:offline ...
Bringing online
Found ram: /sys/devices/system/memory/memory67/state:offline ...
Bringing online
Found ram: /sys/devices/system/memory/memory68/state:offline ...
Bringing online
Found ram: /sys/devices/system/memory/memory69/state:offline ...
Bringing online
Found ram: /sys/devices/system/memory/memory6/state:online ...
Already online
Found ram: /sys/devices/system/memory/memory70/state:offline ...
Bringing online
Found ram: /sys/devices/system/memory/memory71/state:offline ...
Bringing online
Found ram: /sys/devices/system/memory/memory72/state:offline ...
Bringing online
Found ram: /sys/devices/system/memory/memory73/state:offline ...
Bringing online
Found ram: /sys/devices/system/memory/memory74/state:offline ...
Bringing online
Found ram: /sys/devices/system/memory/memory75/state:offline ...
Bringing online
Found ram: /sys/devices/system/memory/memory76/state:offline ...
Bringing online
Found ram: /sys/devices/system/memory/memory77/state:offline ...
Bringing online
Found ram: /sys/devices/system/memory/memory78/state:offline ...
Bringing online
Found ram: /sys/devices/system/memory/memory79/state:offline ...
Bringing online
Found ram: /sys/devices/system/memory/memory7/state:online ...
Already online
Found ram: /sys/devices/system/memory/memory80/state:offline ...
Bringing online
Found ram: /sys/devices/system/memory/memory81/state:offline ...
Bringing online
Found ram: /sys/devices/system/memory/memory82/state:offline ...
Bringing online
Found ram: /sys/devices/system/memory/memory83/state:offline ...
Bringing online
Found ram: /sys/devices/system/memory/memory84/state:offline ...
Bringing online
Found ram: /sys/devices/system/memory/memory85/state:offline ...
Bringing online
Found ram: /sys/devices/system/memory/memory86/state:offline ...
Bringing online
Found ram: /sys/devices/system/memory/memory87/state:offline ...
Bringing online
Found ram: /sys/devices/system/memory/memory88/state:offline ...
Bringing online
Found ram: /sys/devices/system/memory/memory89/state:offline ...
Bringing online
Found ram: /sys/devices/system/memory/memory8/state:online ...
Already online
Found ram: /sys/devices/system/memory/memory90/state:offline ...
Bringing online
Found ram: /sys/devices/system/memory/memory91/state:offline ...
Bringing online
Found ram: /sys/devices/system/memory/memory92/state:offline ...
Bringing online
Found ram: /sys/devices/system/memory/memory93/state:offline ...
Bringing online
Found ram: /sys/devices/system/memory/memory94/state:offline ...
Bringing online
Found ram: /sys/devices/system/memory/memory95/state:offline ...
Bringing online
Found ram: /sys/devices/system/memory/memory96/state:offline ...
Bringing online
Found ram: /sys/devices/system/memory/memory97/state:offline ...
Bringing online
Found ram: /sys/devices/system/memory/memory98/state:offline ...
Bringing online
Found ram: /sys/devices/system/memory/memory99/state:offline ...
Bringing online
Found ram: /sys/devices/system/memory/memory9/state:online ...
Already online
root@ubuntu-template:~# cat /proc/meminfo
MemTotal: 12429660 kB

Script:

#!/bin/bash
# Based on script by William Lam - http://engineering.ucsb.edu/~duonglt/vmware/

# Bring CPUs online
for CPU in $(ls /sys/devices/system/cpu/ |grep -E '(cpu[0-9])')
do
        CPU_DIR="/sys/devices/system/cpu/${CPU}"
        echo "Found cpu: \"${CPU_DIR}\" ..."
        CPU_STATE_FILE="${CPU_DIR}/online"
        if [ -f "${CPU_STATE_FILE}" ]; then
                STATE=$(cat "${CPU_STATE_FILE}" | grep 1)
                if [ "${STATE}" == "1" ]; then
                        echo -e "\t${CPU} already online"
                else
                         echo -e "\t${CPU} is new cpu, onlining cpu ..."
                         echo 1 > "${CPU_STATE_FILE}"
                fi
        else 
                echo -e "\t${CPU} already configured prior to hot-add"
        fi
done

# Bring all new Memory online
for RAM in $(grep line /sys/devices/system/memory/*/state)
do
        echo "Found ram: ${RAM} ..."
        if [[ "${RAM}" == *":offline" ]]; then
                echo "Bringing online"
                echo $RAM | sed "s/:offline$//"|sed "s/^/echo online > /"|source /dev/stdin
        else
                echo "Already online"
        fi
done