A recent systemd update creates an error when resolving DNS requests on virtual machines with Ubuntu Bionic Beaver (version 18.04). Systemd version 237-3ubuntu10.54 contains a bug that causes network connections to drop. The automatic installation of the update on VMs in Azure led to various failures. The Azure Kubernetes Service (AKS) in particular is severely affected globally. The Ubuntu and Azure teams are working to fix the bug and its impact. The problem is limited to Ubuntu 18.04.
The Canonical / Ubuntu team names two possible workarounds on Launchpad. Affected users either restart their instances or enter udevadm trigger -cadd -yeth0 && systemctl restart systemd-networkd as root. As a starting point, the developers identified open-vm-tools, from which the udevam trigger comes. Due to the way the driver match is set up in Azure for networking, the key-value pair is lost when running the udevam trigger. This leads to the loss of DNS information from the server.
The network failure of Ubuntu VMs running Bionic Beaver is still causing problems with Kubernetes and containers in Azure globally. The figure shows the affected regions in Europe.
Same problem two years ago
At Microsoft, the Azure team is working to restore the Kubernetes Service. Automatic recovery was able to repair most of the clusters, but individual nodes are still resisting, according to Azure. The systemd update on Azure is suspended until a solution is found. According to the Ubuntu team, the error appeared two years ago as bug 1902960.
The current situation at Azure can be found on the status page, the Ubuntu report on Launchpad.
To home page