Canonical
on 20 December 2017
This article originally appeared on the Ubuntu On Big Iron blog
This article mainly assumes that an Ubuntu Server 17.10 (artful) is installed in LPAR.
Ubuntu 17.10 comes with a qemu-kvm version that is new enough to act as a KVM host that also allows to PXE netboot KVM virtual machines.
Any potentially different or additional steps for Ubuntu Server 16.04(.x) are marked in italics.
PXE boot will only work on Ubuntu 16.04 if the qemu-kvm package from the Pike Ubuntu Cloud Archive (UCA) is used.
Prepare virtualization packages
Note:
On Ubuntu 16.04.x qemu-kvm from the Pike Ubuntu Cloud Archive (UCA) need to be installed.
The Pike Cloud Archive can be enabled like this:
$ sudo add-apt-repository cloud-archive:pike $ sudo apt update
On Ubuntu 17.10 (or higher) just install the default qemu-kvm and related packages with:
$ sudo apt install -qq -y libvirt-daemon-system qemu-kvm
The just installed packages modify the underlying environment, hence it’s recommended to quickly logout and login again to make the changes take effect.
Prepare s390 netboot ROM
$ sudo apt -y -qq install ubuntu-dev-tools $ pull-lp-source qemu artful pull-lp-source: Downloading qemu version 1:2.10+dfsg-0ubuntu3.1 ... $ sudo cp qemu-2.10+dfsg/pc-bios/s390-netboot.img /usr/share/qemu/
Prepare a tftp server
Now a tftp server is needed for PXE netboot, so let’s install and configure tftp:
$ sudo apt -y -qq install tftpd xinetd tftp
...
$ cat << EOF | sudo tee /etc/xinetd.d/tftp
service tftp
{
protocol = udp
port = 69
socket_type = dgram
wait = yes
user = nobody
server = /usr/sbin/in.tftpd
server_args = /srv/tftp
disable = no
}
EOF
Create the tftp folder, copy over the pxelinux.0 file, adjust the folder options and (re-)start the service:
$ sudo mkdir -p /srv/tftp $ sudo chmod -R 777 /srv/tftp $ sudo chown -R nobody: /srv/tftp $ sudo systemctl restart xinetd
By now you should be able to check if tftp is listening to your local port 69 on the default libvirt networks IP of your host:
$ nc -uvz 192.168.122.1 69 Connection to 10.245.236.13 69 port [udp/tftp] succeeded!
Prepare libvirt network to provide PXE
Since we have slightly different network needs, we will modify the default network a bit,
which was created for you on install. In this step we add the lines in bold:
It's not too bad to do a quick backup of the virtual network configuration at the beginning:
$ virsh net-dumpxml default > net-default.xml
Now do the needed tftp and bootp modifications:
$ virsh net-edit default <network> <name>default</name> <uuid>cfa3d2de-4c67-4f8f-91eb-69502e59e64d</uuid> <forward mode='nat'/> <bridge name='virbr0' stp='on' delay='0'/> <mac address='52:54:00:08:6f:33'/> <ip address='192.168.122.1' netmask='255.255.255.0'> <strong><tftp root='/srv/tftp'/></strong> <dhcp> <range start='192.168.122.2' end='192.168.122.254'/> <strong><bootp file='pxelinux.0' server='192.168.122.1'/></strong> </dhcp> </ip> </network> </pre>
And restart the virtual network again:
$ virsh net-destroy default Network default destroyed $ virsh net-start default Network default started
Prepare netboot image
Before we can perform a PXE netboot, a 'pxelinux.0' netboot image is needed.
We will now create a pxelinux.0 file that is based on the Ubuntu 16.04.3 (xenial) installer.
Now, download the latest xenial installer files (kernel and initrd):
$ wget --quiet http://ports.ubuntu.com/ubuntu-ports/dists/xenial-updates/main/installer-s390x/current/images/generic/kernel.ubuntu http://ports.ubuntu.com/ubuntu-ports/dists/xenial-updates/main/installer-s390x/current/images/generic/initrd.ubuntu
And create a pxelinux.0 file out of these kernel and initrd installer files:For the PXE image creation a separate tool is needed from the s390-tools collection:
$ git clone --quiet https://github.com/ibm-s390-tools/s390-tools.git ... $ ./s390-tools/netboot/mk-s390image kernel.ubuntu -r initrd.ubuntu pxelinux.0 $ ll pxelinux.0 -rw-rw-r-- 1 ubuntu ubuntu 15346314 Dec 5 05:12 pxelinux.0 $ sudo cp pxelinux.0 /srv/tftp $ sudo chmod -R 777 /srv/tftp $ sudo chown -R nobody: /srv/tftp
It's now time for a little download test:
$ cd /tmp/ $ echo 'get pxelinux.0' | tftp 192.168.122.1 tftp> Received 15453177 bytes in 0.3 seconds $ cd -
Prepare guest definition
Now proceed with the creation of a KVM virtual machine definition for the PXE netboot VM:
$ cat << EOF > netboot.xml <domain type='kvm'> <name>netboot</name> <memory unit='KiB'>2097152</memory> <vcpu placement='static'>1</vcpu> <os> <type arch='s390x' machine='s390-ccw-virtio-xenial'>hvm</type> <boot dev='network'/> </os> <devices> <emulator>/usr/bin/kvm</emulator> <disk type='file' device='disk'> <driver name='qemu' type='qcow2'/> <source file='/var/lib/libvirt/images/netboot.qcow2'/> <target dev='vda' bus='virtio'/> </disk> <interface type='network'> <source network='default'/> <model type='virtio'/> </interface> <console type='pty'> <target type='sclp' port='0'/> </console> </devices> </domain> EOF
Let's also quickly create a virtual disk that is already part of the VM definition above:
$ sudo qemu-img create -f qcow2 /var/lib/libvirt/images/netboot.qcow2 5G Formatting '/var/lib/libvirt/images/netboot.qcow2', fmt=qcow2 size=5368709120 cluster_size=65536 lazy_refcounts=off refcount_bits=16
And now define the virtual machine named 'netboot':
$ virsh define netboot.xml Domain netboot defined from netboot.xml
Finally start the netboot guest to prove that PXE netboot is generally working:
$ virsh start netboot --console
Domain netboot started
Connected to domain netboot
Escape character is ^]
Using MAC address: 52:54:00:fa:01:01
Requesting information via DHCP: done
Using IPv4 address: 192.168.122.217
Requesting file "pxelinux.0" via TFTP from 192.168.122.1
Receiving data: 14986 KBytes
TFTP: Received pxelinux.0 (14986 KBytes)
Network loading done, starting kernel...
[ 0.395926] Initializing cgroup subsys cpuset
[ 0.395929] Initializing cgroup subsys cpu
[ 0.395930] Initializing cgroup subsys cpuacct
[ 0.395933] Linux version 4.4.0-96-generic (buildd@z13-009) (gcc version 5.4.0 20160609 (Ubuntu 5.4.0-6ubuntu1~16.04.4) ) #119-Ubuntu SMP Tue Sep 12 14:59:56 UTC 2017 (Ubuntu 4.4.0-96.119-generic 4.4.83)
[ 0.395937] setup: Linux is running under KVM in 64-bit mode
...
[ 0.659971] AppArmor: AppArmor sha1 policy hashing enabled
[ 0.659975] ima: No TPM chip found, activating TPM-bypass!
┌──────────────────┤ [!!] Configure the network device ├──────────────────┐
│ │
│ Please choose the type of your primary network interface that you │
│ will need for installing the Debian system (via NFS or HTTP). Only │
│ the listed devices are supported. │
│ │
│ Network device type: │
│ │
│ ctc: Channel to Channel (CTC) or ESCON connection │
│ qeth: OSA-Express in QDIO mode / HiperSockets │
│ iucv: Inter-User Communication Vehicle - available for VM guests │
│ virtio: KVM VirtIO │
│ │
│ │
│ │
└─────────────────────────────────────────────────────────────────────────┘
For now let's leave the VM's console via Ctrl+]
$
The same again but with Preseeds
Prepare preseed
Okay, this first virtual machine was able to netboot into the default installer - but that's just the first step.
The second step is to combine this with a non-interactive preseed installation.
To do so download an example pressed file (like this - rename it to 'preseed.cfg') and store it locally in the tftp directory:
$ wget --quiet "https://wiki.ubuntu.com/S390X/InstallationGuide/AutomatedInstallsWithPreseed?action=AttachFile&do=get⌖=preseed_kvm.cfg" -O /srv/tftp/preseed.cfg
Do another quick check if preseed.cfg is reachable via tftp:
$ cd /tmp /tmp$ echo 'get preseed.cfg' | tftp 192.168.122.1 tftp> Received 699 bytes in 0.0 seconds $ cd -
Prepare pxe config to map to preseed config
A pxelinux.cfg/default configuration file need to be created as well - just to be able to pass over the kernel parameters for preseed installation.
$ sudo mkdir -p /srv/tftp/pxelinux.cfg $ cat << EOF | sudo tee /srv/tftp/pxelinux.cfg/default kernel kernel.ubuntu initrd initrd.ubuntu append auto=true priority=critical url=tftp://192.168.122.1/preseed.cfg s390-netdevice/choose_networktype=virtio netcfg/disable_dhcp=false netcfg/get_hostname=netboot-preseed netcfg/get_domain=mydomain.com network-console/password=pass4inst network-console/start=true EOF
Now some additional tools/packages are needed to generate a general pxelinux.0 file:
$ sudo apt install -y -qq kexec-tools gcc make $ cd s390-tools/netboot $ make -f Makefile.pxelinux.0 $ mv /srv/tftp/pxelinux.0 /srv/tftp/pxelinux.0.installer $ ./s390-tools/netboot/mk-s390image ./kernel.ubuntu pxelinux.0
Now we make all this available to PXE boot:
- the new pxelinux.0 that will read the initial config
- the kernel and initrd to be loaded from there and to kexec into
$ cp pxelinux.0 /srv/tftp/pxelinux.0 $ cd - $ cp initrd.ubuntu /srv/tftp/ $ cp kernel.ubuntu /srv/tftp/ $ sudo chmod -R 777 /srv/tftp $ sudo chown -R nobody: /srv/tftp
Create second VM
Now define a second guest named 'netboot-preseed' for the PXE netboot initiated non-interactive preseed installation on KVM s390x.
It will be done based on the previous example, hence we only need to adjust some names:
$ sed -e 's/netboot/netboot-preseed/g' netboot.xml > netboot-preseed.xml
Again create the virtual disk - in this case really needed, because the installation should complete:
$ sudo qemu-img create -f qcow2 /var/lib/libvirt/images/netboot-preseed.qcow2 5G
Now define that second VM:
$ virsh define netboot-preseed.xml Domain netboot-preseed defined from netboot-preseed.xml
Now start this second VM:
$ virsh start netboot-preseed --console
Domain netboot-preseed started
Connected to domain netboot-preseed
Escape character is ^]
Using MAC address: 52:54:00:fa:4d:4d
Requesting information via DHCP: 010
...
Requesting information via DHCP: done
Using IPv4 address: 192.168.122.213
Requesting file "pxelinux.0" via TFTP from 192.168.122.1
Receiving data: 14986 KBytes
TFTP: Received pxelinux.0 (14986 KBytes)
Network loading done, starting kernel...
[ 0.394899] Initializing cgroup subsys cpuset
[ 0.394902] Initializing cgroup subsys cpu
[ 0.394903] Initializing cgroup subsys cpuacct
[ 0.394907] Linux version 4.4.0-96-generic (buildd@z13-009) (gcc version 5.4.0 20160609 (Ubuntu 5.4.0-6ubuntu1~16.04.4) ) #119-Ubuntu SMP Tue Sep 12 14:59:56 UTC 2017 (Ubuntu 4.4.0-96.119-generic 4.4.83)
[ 0.394912] setup: Linux is running under KVM in 64-bit mode
...
can't open /dev/tty3: No such device or address
can't open /dev/tty2: No such device or address
can't open /dev/tty4: No such device or address ⇐ just ignore these lines !
...
Finishing the installation
│ 95% │
The system is going down NOW!
Sent SIGTERM to all processes device or address
Sent SIGKILL to all processes device or
$
The above was the post install shutdown, last task is to adjust the boot order (to not install again):
$ virsh edit netboot
and change:
<boot dev='network'/>
to:
<boot dev='hd'/>
and to restart - this time from disk:
No do a final restart of the VM and have fun:
$ virsh start netboot-preseed --console [...] netboot-preseed login: _
At this stage two KVM virtual machine are in use. The first one ('netboot') was netboot-ed and landed in a regular interactive d-i installation; the second ('netboot-preseed') was again net-booted, but then performed a non-interactive d-i preseed installation.
