Creating Ubuntu 20.04 KVM Virtual Machines with virt-install and virsh

In the previous chapter we explored the creation of KVM guest operating systems on an Ubuntu host using Cockpit and the virt-manager graphical tool. In this chapter we will turn our attention to the creation of KVM-based virtual machines using the virt-install and virsh command-line tools. These tools provide all the capabilities of the virt-manager and Cockpit options with the added advantage that they can be used within scripts to automate virtual machine creation. In addition, the virsh command allows virtual machines to be created based on a specification contained within a configuration file.

The virt-install tool is supplied to allow new virtual machines to be created by providing a list of command-line options. This chapter assumes that the necessary KVM tools are installed. For details on these requirements read the chapter entitled “Installing KVM Virtualization on Ubuntu”.

1.1  Running virt-install to build a KVM Guest System

The virt-install utility accepts a wide range of command-line arguments that are used to provide configuration information related to the virtual machine being created. Some of these commandline options are mandatory (specifically name, memory and disk storage must be provided) while others are optional.

At a minimum, a virt-install command will typically need the following arguments:

  • –name – The name to be assigned to the virtual machine.
  • –memory – The amount of memory to be allocated to the virtual machine.
  • –disk – The name and location of an image file to be used as storage for the virtual machine. This file will be created by virt-install during the virtual machine creation unless the
  • –import option is specified to indicate an existing image file is to be used.
  • –cdrom or –location – Specifies the local path or the URL of a remote ISO image containing the installation media for the guest operating system.

A summary of all the arguments available for use when using virt-install can be found in the man page:

$ man virt-install

1.2  An Example Ubuntu virt-install Command

With reference to the above command-line argument list, we can now look at an example command-line construct using the virt-install tool.

Note that in order to be able to display the virtual machine and complete the installation, a virtviewer instance will need to be connected to the virtual machine after it is started by the virt-install utility. By default, virt-install will attempt to launch virt-viewer automatically once the virtual machine starts running. If virt-viewer is not available, virt-install will wait until a virt-viewer connection is established. The virt-viewer session may be running locally on the host system if it has a graphical desktop, or a connection may be established from a remote client as outlined in the chapter entitled “Creating KVM Virtual Machines using Cockpit and virt-manager”.

The following command creates a new KVM virtual machine configured to run Fedora using KVM para-virtualization. It creates a new 10GB disk image, assigns 1024MB of RAM to the virtual machine and configures a virtual CD device for the installation media ISO image:

# virt-install --name MyFedora --memory 1024 --disk path=/tmp/myFedora.img,size=10 --network network=default --os-variant fedora28 --cdrom /tmp/Fedora-Server-dvd-x86_64.iso

As the creation process runs, the virt-install command will display status updates of the creation progress:

Starting install...
Allocating 'MyFedora.img'                                |  10 GB  00:00:01     
Domain installation still in progress. Waiting for installation to complete.

Once the guest system has been created, the virt-viewer screen will appear containing the operating system installer loaded from the specified installation media:

Figure 27-1

From this point, follow the standard installation procedure for the guest operating system.

1.3  Starting and Stopping a Virtual Machine from the Command-Line

Having created the virtual machine from the command-line it stands to reason that you may also need to start it from the command-line in the future. This can be achieved using the virsh command-line utility, referencing the name assigned to the virtual machine during the creation process. For example:

# virsh start MyFedora

Similarly, the virtual machine may be sent a shutdown signal as follows:

# virsh shutdown MyFedora

If the virtual machine fails to respond to the shutdown signal and does not begin a graceful shutdown the virtual machine may be destroyed (with the attendant risks of data loss) using the destroy directive:

# virsh destroy MyFedora

1.4  Creating a Virtual Machine from a Configuration File

The virsh create command can take as an argument the name of a configuration file on which to base the creation of a new virtual machine. The configuration file uses XML format. Arguably the easiest way to create a configuration file is to dump out the configuration of an existing virtual machine and modify it for the new one. This can be achieved using the virsh dumpxml command. The following command outputs the configuration data for a virtual machine domain named MyFedora to a file named MyFedora.xml:

# virsh dumpxml MyFedora > MyFedora.xml

Once the file has been generated, load it into an editor to review and change the settings for the new virtual machine.

At the very least, the <name>, <uuid> and image file path <source file> must be changed in order to avoid conflict with the virtual machine from which the configuration was taken. In the case of the UUID, this line can simply be deleted from the file.

The virtualization type, memory allocation and number of CPUs to name but a few options may also be changed if required. Once the file has been modified, the new virtual machine may be created as follows:

# virsh create MyFedora.xml

1.5  Summary

KVM provides the virt-install and virsh command-line tools as a quick and efficient alternative to using the Cockpit and virt-manager tools to create and manage virtual machine instances. These tools have the advantage that they can be used from within scripts to automate the creation and management of virtual machines. The virsh command also includes the option to create VM instances from XML-based configuration files.

Creating Ubuntu 20.04 KVM Virtual Machines using Cockpit and virt-manager

KVM-based virtual machines can easily be configured on Ubuntu using either the virt-install command-line tool, the virt-manager GUI tool or the Virtual Machines module of the Cockpit web console. For the purposes of this chapter we will use Cockpit and the virt-manager tool to install a Fedora distribution as a KVM guest on an Ubuntu host.

The command-line approach to virtual machine creation will be covered in the next chapter entitled “Creating KVM Virtual Machines with virt-install and virsh”.

1.1  Installing the Cockpit Virtual Machines Module

By default, the virtual machines module may not be included in a standard Cockpit installation. Assuming that Cockpit is installed and configured, the virtual machines module may be installed as follows:

# apt install cockpit-machines

Once installed, the Virtual Machines option (marked A in Figure 26-1) will appear in the navigation panel next time you log into the Cockpit interface:

Figure 26-1

1.2  Creating a Virtual Machine in Cockpit

To create a virtual machine in Cockpit, simply click on the Create VM button marked B in Figure 26-1 to display the creation dialog.

Within the dialog, enter a name for the machine and choose whether the installation media is in the form of an ISO accessible via a URL or a local filesystem path. Ideally, also select the vendor and operating system type information for the guest. While not essential, this will aid the system in optimizing the virtual machine for the guest.

Also specify the size of the virtual disk drive to be used for the operating system installation and the amount of memory to be allocated to the virtual machine:

Figure 26-2

Note that Cockpit provides the choice of running the guest with a Session or System connection. If the system option is selected, the guest will connect to the system instance of the libvirtd service which is already running in the background with root privileges. The session option, however, starts a new libvirtd service that is owned by the current user and then connects the host to it. A session guest will, by default, use a storage pool that is local to the user’s account (for example / home/demo/.local/share/libvirt/images) and will be accessible only to the owner. A system session, on the other hand will be accessible to all users with appropriate privileges and will, by default, use storage located in /var/lib/libvirt/images.

For this example, select the System option, leave the Immediately Start VM option unselected and, once the new virtual machine has been configured, click on the Create button to build the virtual machine. After the creation process is complete, the new VM will appear in Cockpit as shown in Figure 26-3:

Figure 26-3

As described in “An Overview of Virtualization Techniques”, KVM provides virtual machines with a number of options in terms of network configuration. To view and change the network settings of a virtual machine, click on the Network interfaces tab as shown in Figure 26-4 followed by the Edit button located next to the network entry:

Figure 26-4

In the resulting dialog, the Network Type menu may be used to change the type of network connection, for example from virtual network (NAT) to direct (MacVTap).

1.3  Starting the Installation

To start the new virtual machine and begin installing the guest operating system from the designated installation media, click on the Install button highlighted in Figure 26-3 above. Cockpit will start the virtual machine and switch to the Consoles view where the guest OS screen will appear:

Figure 26-5

If the installation fails, check the message to see if it reads as follows:

unsupported configuration: CPU mode ‘custom’ for x86_64 kvm domain on x86_64 host is not supported by hypervisor

To resolve this issue, delete the newly created virtual machine, reboot the system and then recreate the machine.

Alternatively, check whether the message reads as follows:

Could not open ‘<path to iso image>’: Permission denied Domain installation does not appear to have been successful.

This usually occurs because the QEMU emulator runs as a user named qemu which does not have access to the directory in which the ISO installation image is located. To resolve this issue, open a terminal window (or connect with SSH if the system is remote), change directory to the location of the ISO image file and add the qemu user to the access control list (ACL) of the parent directory as follows:

# cd /path/to/iso/directory
# setfacl --modify u:qemu:x ..

After making this change, check the setting as follows:

# getfacl ..
# file: ..
# owner: demo
# group: demo
user::rwx
user:qemu:--x
group::---
mask::--x
other::---

Once these changes have been made, click on the Install button once again to complete the installation.

To complete the installation, interact with the screen in the Consoles view just as you would if you were installing the operating system on physical hardware.

It is also possible to connect with and display the graphical console for the VM from outside the Cockpit browser session using the virt-viewer tool. To install virt-viewer on an Ubuntu system, run the following command:

# apt install virt-viewer

The virt-viewer tool is also available for Windows systems and can be downloaded from the following URL:

https://virt-manager.org/download/

To connect with a virtual machine running on the local host, simply run virt-viewer and select the virtual machine to which you wish to connect from the resulting dialog:

Figure 26-6

The above command will list system-based virtual machines. To list and access session-based guests, launch virt-viewer as follows:

$ virt-viewer --connect qemu:///session

Alternatively, it is also possible to specify the virtual machine name and bypass the selection dialog entirely, for example:

$ virt-viewer myFedoraGuest
$ virt-viewer --connect qemu:///session myFedoraGuest

To connect a virt-viewer instance to a virtual machine running on a remote host using SSH, the following command can be used:

$ virt-viewer --connect qemu+ssh://<user>@<host>/system <guest name>

For example:

$ virt-viewer --connect qemu+ssh://root@192.168.1.122/system MyFedoraGuest

When using this technique it is important to note that you will be prompted twice for the user password before the connection will be fully established.

Once the virtual machine has been created, the Cockpit interface can be used to monitor the machine and perform tasks such as rebooting, shutting down or deleting the guest system. An option is also included on the Disks panel to add additional disks to the virtual machine configuration.

1.4  Working with Storage Volumes and Storage Pools

When a virtual machine is created it will usually have associated with it at least one virtual disk drive. The images that represent these virtual disk drives are stored in storage pools. A storage pool can take the form of an existing directory on a local filesystem, a filesystem partition, physical disk device, Logical Volume Management (LVM) volume group or even a remote network file system (NFS).

Each storage pool is divided into one or more storage volumes. Storage volumes are typically individual image files, each representing a single virtual disk drive, but can also take the form of physical disk partitions, entire disk drives or LVM volume groups.

When a virtual machine was created using the previous steps, a default storage pool was created into which virtual machine images may be stored. This default storage pool occupies space on the root filesystem and can be reviewed from within the Cockpit Virtual Machine interface by selecting the Storage Pools option at the top of the panel marked C in Figure 26-1 above.

When selected, the screen shown in Figure 26-7 below will appear containing a list of storage pools currently configured on the system:

Figure 26-7

In the above example, the default storage pool is located on the root filesystem and stores the virtual machine image in the /var/lib/libvirtd/images directory. To view the storage volumes contained within the pool, select the Storage Volumes tab highlighted in Figure 26-8:

Figure 26-8

In the case of the Fedora guest, the storage volume takes the form of an image file named myFedoraGuest.qcow2. To find out which storage volume a particular virtual machine uses, return to the main Virtual Machine Cockpit screen, select the virtual machine and display the Disks panel as shown in Figure 26-9:

Figure 26-9

Although using the default storage pool is acceptable for testing purposes and early experimentation, it is recommended that additional pools be created for general virtualization use. To create a new storage pool, display the Storage Pools screen within Cockpit and click on the Create New Storage Pool button to display the dialog shown in Figure 26-10:

Figure 26-10

In the above example, a new storage pool is being created named MyPool using a file system partition mounted as /MyPool within the local filesystem (the topic of disk drives, partitions and mount points is covered later in the chapter entitled “Adding a New Disk Drive to an Ubuntu System”). Once created, the pool will now be listed within the Cockpit storage pool screen and can be used to contain storage volumes as new virtual machines are created.

At the time of writing, it was not possible to create a new storage volume within a custom storage pool from within the Cockpit interface. It is, however, possible to do this from within the Virtual Machine manager as outlined in the following section.

1.5  Creating a Virtual Machine using virt-manager

With the caveat that virt-manager may one day be discontinued once the Virtual Machines Cockpit extension is fully implemented, the remainder of this chapter will explore the use of this tool to create new virtual machines.

1.6  Starting the Virtual Machine Manager

Begin by launching Virtual Machine Manager from the command-line in a terminal window by running virt-manager. Once loaded, the virtual machine manager will prompt for the password of the currently active user prior to displaying the following screen:

Figure 26-11

The main screen lists the current virtual machines running on the system. By default the manager should be connected to the system libvirtd instance. If it is not, connect to the host system by right-clicking on the entry in the list and selecting Connect from the popup menu. To manage session-based virtual machines, select the File -> Add Connection… menu option to display the dialog shown in Figure 26-12:

Figure 26-12

Within this dialog, select QEMU/KVM user session from the Hypervisor menu and click on the Connect button. On returning to the main virt-manager screen, the user session hypervisor should now be listed:

Figure 26-13

To create a new virtual system, click on the new virtual machine button (the far left button on the toolbar) or right-click on the hypervisor entry and select New from the resulting menu to display the first screen of the New VM wizard. In the Name field enter a suitably descriptive name for the virtual system. On this screen, also select the location of the media from which the guest operating system will be installed. This can either be a CD or DVD drive, an ISO image file accessible to the local host, a network install using HTTP, FTP, NFS or PXE or the disk image from an existing virtual machine:

Figure 26-14

1.7  Configuring the KVM Virtual System

Clicking Forward will display a screen seeking additional information about the installation process. The screen displayed and information required will depend on selections made in the preceding screen. For example, if a CD, DVD or ISO was selected, this screen will ask for the specific location of the ISO file or physical media device. This screen also attempts to identify the type and version of the guest operating system to be installed (for example the Windows version or Linux distribution) based on the installation media specified. If it is unable to do so, uncheck the Automatically detect from installation media / source option, type in the first few characters of the operating system name and select an option from the list of possible matches:

Figure 26-15

Once these settings are complete, click the Forward button to configure CPU and memory settings. The optimal settings will depend on the number of CPUs and amount of physical memory present in the host together with the requirements of other applications and virtual machines that will run in parallel with the new virtual machine:

Figure 26-16

On the next screen, options are available to create an image disk of a specified size, select a preexisting volume or to create a storage volume of a specified format (raw, vmdk, ISO etc). Unless you have a specific need to use a particular format (for example you might need to use vmdk to migrate to a VMware based virtualization environment at a later date) or need to use a dedicated disk or partition, it is generally adequate to simply specify a size on this screen:

Figure 26-17

If the default settings are used here, the virtual machine will use a storage volume within the default storage pool for the virtual disk drive. To make use of the custom “MyPool” storage pool created earlier in the chapter, enable the Select or create custom storage option before clicking on the Manage… button.

In the storage volume dialog, select the MyPool entry in the left hand panel, followed by the + button in the main panel to create a new storage volume:

Figure 26-18

Note that the + button located in the bottom left-hand corner of the dialog may also be used to create new storage pools as an alternative to using the Cockpit interface.

In the configuration screen (Figure 26-19), name the storage volume, select the volume size and click on the Finish button to create the volume and assign it to the virtual machine:

Figure 26-19

Once these settings are configured, select the new volume and click on the Choose Volume button. Click the Forward button once more. The final screen displays a summary of the configuration. Review the information displayed. Advanced options are also available to change the virtual network configuration for the guest as shown in Figure 26-20:

Figure 26-20

1.8  Starting the KVM Virtual Machine

Click on the Finish button to begin the creation process. The virtualization manager will create the disk and configure the virtual machine before starting the guest system. The new virtual machine will appear in the main virt-manager window with the status set to Running as illustrated in Figure 26-21:

Figure 26-21

By default, the console for the virtual machine should appear in the virtual machine viewer window. To view the console of the running machine at any future time, ensure that it is selected in the virtual machine list and select the Open button from the toolbar. The virtual machine viewer should be ready for the installation process to begin:

Figure 26-22

From this point on, simply follow the operating system installation instructions to install the guest OS in the KVM virtual machine.

1.9 Summary

This chapter has outlined two different ways to create new KVM-based virtual machines on an Ubuntu host system. The first option covered involves the use of the Cockpit web-based interface to create and manage virtual machines. This has the advantage of not requiring access to a desktop environment running on the host system. An alternative option is to use the virtmanager graphical tool. With these basics covered, the next chapter will cover the creation of virtual machines from the command-line.

Installing KVM Virtualization on Ubuntu 20.04

Earlier versions of Ubuntu provided two virtualization platforms in the form of Kernel-based Virtual Machine (KVM) and Xen. In recent releases, support for Xen has been removed leaving KVM as the only bundled virtualization option supplied with Ubuntu. In addition to KVM, third party solutions are available in the form of products such as VMware and Oracle VirtualBox. Since KVM is supplied with Ubuntu, however, this is the virtualization solution that will be covered in this and subsequent chapters.

Before plunging into installing and running KVM it is worth taking a little time to talk about how it fits into the various types of virtualization outlined in the previous chapter.

1.1  An Overview of KVM

KVM is categorized as a Type-1 hypervisor virtualization solution that implements full virtualization with support for unmodified guest operating systems using Intel VT and AMD-V hardware virtualization support.

KVM differs from many other Type-1 solutions in that it turns the host Linux operating system itself into the hypervisor, allowing bare metal virtualization to be implemented while still running a full, enterprise level host operating system.

1.2  KVM Hardware Requirements

Before proceeding with this chapter we need to take a moment to discuss the hardware requirements for running virtual machines within a KVM environment. First and foremost, KVM virtualization is only available on certain processor types. As previously discussed, these processors must include either Intel VT or AMD-V technology.

To check for virtualization support, run the following command in a terminal window:

# lscpu | grep Virtualization:

If the system contains a CPU with Intel VT support, the above command will provide the following output:

Virtualization: VT-x

Alternatively, the following output will be displayed when a CPU with AMD-V support is detected:

Virtualization: AMD-V

If the CPU does not support virtualization, no output will be displayed by the above lscpu command.

Note that while the above commands only report whether the processor supports the respective feature, it does not indicate whether the feature is currently enabled in the BIOS. In practice virtualization support is typically disabled by default in the BIOS of most systems. It is recommended, therefore, that you check your BIOS settings to ensure the appropriate virtualization technology is enabled before proceeding with this tutorial.

Unlike a dual booting environment, a virtualized environment involves the running of two or more complete operating systems concurrently on a single computer system. This means that the system must have enough physical memory, disk space and CPU processing power to comfortably accommodate all these systems in parallel. Before beginning the configuration and installation process check on the minimum system requirements for both Ubuntu and your chosen guest operating systems and verify that your host system has sufficient resources to handle the requirements of both systems.

1.3  Preparing Ubuntu for KVM Virtualization

Unlike Xen, it is not necessary to run a special version of the kernel in order to support KVM. As a result KVM support is already available for use with the standard kernel via the installation of a KVM kernel module, thereby negating the need to install and boot from a special kernel. To avoid conflicts, however, if a Xen enabled kernel is currently running on the system, reboot the system and select a non-Xen kernel from the boot menu before proceeding with the remainder of this chapter.

The tools required to setup and maintain a KVM-based virtualized system are not installed by default unless specifically selected during the Ubuntu operating system installation process. To install the KVM tools from the command prompt, execute the following command in a terminal window:

# apt install qemu-kvm libvirt-clients libvirt-daemon-system bridge-utils

If you have access to a graphical desktop environment the virt-manager package is also recommended:

# apt install virt-manager

1.4  Verifying the KVM Installation

It is worthwhile checking that the KVM installation worked correctly before moving forward. When KVM is installed and running, two modules will have been loaded into the kernel. The presence or otherwise of these modules can be verified in a terminal window by running the following command:

# lsmod | grep kvm

Assuming that the installation was successful the above command should generate output similar to the following:

# lsmod | grep kvm
kvm_intel             237568  0
kvm                   737280  1 kvm_intel
irqbypass              16384  1 kvm

Note that if the system contains an AMD processor the kvm module will likely read kvm_amd rather than kvm_intel.

The installation process should also have configured the libvirtd daemon to run in the background. Once again using a terminal window, run the following command to ensure libvirtd is running:

# systemctl status libvirtd
 libvirtd.service - Virtualization daemon
   Loaded: loaded (/usr/lib/systemd/system/libvirtd.service; enabled; vendor preset: enabled)
   Active: active (running) since Wed 2019-03-06 14:41:22 EST; 3min 54s ago

If the process is not running, it may be started as follows:

# systemctl enable --now libvirtd
# systemctl start libvirtd

If the desktop environment is available, run the virt-manager tool by selecting Activities and entering “virt” into the search box. When the Virtual Machine Manager icon appears, click on it to launch it. When loaded, the manager should appear as illustrated in the following figure:

Figure 25-1

If the QEMU/KVM entry is not listed, select the File -> Add Connection menu option and, in the resulting dialog, select the QEMU/KVM Hypervisor before clicking on the Connect button:

Figure 25-2

If the manager is not currently connected to the virtualization processes, right-click on the entry Installing KVM Virtualization on Ubuntu listed and select Connect from the popup menu.

1.5  Summary

KVM is a Type-1 hypervisor virtualization solution that implements full virtualization with support for unmodified guest operating systems using Intel VT and AMD-V hardware virtualization support. It is the default virtualization solution bundled with Ubuntu and can be installed quickly and easily on any Ubuntu system with appropriate processor support. With KVM support installed and enabled, the next few chapters will outline some of the options for installing and managing virtual machines on an Ubuntu host.

An Overview of Ubuntu 20.04 Virtualization Techniques

Virtualization is generically defined as the ability to run multiple operating systems simultaneously on a single computer system. While not necessarily a new concept, Virtualization has come to prominence in recent years because it provides a way to fully utilize the CPU and resource capacity of a server system while providing stability (in that if one virtualized guest system crashes, the host and any other guest systems continue to run).

Virtualization is also useful in terms of trying out different operating systems without having to configure dual boot environments. For example, you can run Windows in a virtual machine without having to re-partition the disk, shut down Ubuntu and then boot from Windows. You simply start up a virtualized version of Windows as a guest operating system. Similarly, virtualization allows you to run other Linux distributions from within an Ubuntu system, providing concurrent access to both operating systems.

When deciding on the best approach to implementing virtualization it is important to have a clear understanding of the different virtualization solutions that are currently available. The purpose of this chapter, therefore, is to describe in general terms the virtualization techniques in common use today.

1.1  Guest Operating System Virtualization

Guest OS virtualization, also referred to as application-based virtualization, is perhaps the easiest concept to understand. In this scenario the physical host computer system runs a standard unmodified operating system such as Windows, Linux, UNIX or macOS. Running on this operating system is a virtualization application which executes in much the same way as any other application such as a word processor or spreadsheet would run on the system. It is within this virtualization application that one or more virtual machines are created to run the guest operating systems on the host computer.

The virtualization application is responsible for starting, stopping and managing each virtual machine and essentially controlling access to physical hardware resources on behalf of the individual virtual machines. The virtualization application also engages in a process known as binary rewriting which involves scanning the instruction stream of the executing guest system and replacing any privileged instructions with safe emulations. This has the effect of making the guest system think it is running directly on the system hardware, rather than in a virtual machine within an application.

The following figure provides an illustration of guest OS based virtualization:

Figure 24-1

As outlined in the above diagram, the guest operating systems operate in virtual machines within the virtualization application which, in turn, runs on top of the host operating system in the same way as any other application. Clearly, the multiple layers of abstraction between the guest operating systems and the underlying host hardware are not conducive to high levels of virtual machine performance. This technique does, however, have the advantage that no changes are necessary to either host or guest operating systems and no special CPU hardware virtualization support is required.

1.2  Hypervisor Virtualization

In hypervisor virtualization, the task of a hypervisor is to handle resource and memory allocation for the virtual machines in addition to providing interfaces for higher level administration and monitoring tools. Hypervisor based solutions are categorized as being either Type-1 or Type-2.

Type-2 hypervisors (sometimes referred to as hosted hypervisors) are installed as software applications that run on top of the host operating system, providing virtualization capabilities by coordinating access to resources such as the CPU, memory and network for guest virtual machines. Figure 24-2 illustrates the typical architecture of a system using Type-2 hypervisor virtualization:

Figure 24-2

To understand how Type-1 hypervisors work, it helps to understand a little about Intel x86 processor architecture. The x86 family of CPUs provides a range of protection levels known as rings in which code can execute. Ring 0 has the highest level privilege and it is in this ring that the operating system kernel normally runs. Code executing in ring 0 is said to be running in system space, kernel mode or supervisor mode. All other code such as applications running on the operating system operate in less privileged rings, typically ring 3.

In contrast to Type-2 hypervisors, Type-1 hypervisors (also referred to as metal or native hypervisors) run directly on the hardware of the host system in ring 0. Clearly, with the hypervisor occupying ring 0 of the CPU, the kernels for any guest operating systems running on the system must run in less privileged CPU rings. Unfortunately, most operating system kernels are written explicitly to run in ring 0 for the simple reason that they need to perform tasks that are only available in that ring, such as the ability to execute privileged CPU instructions and directly manipulate memory. A number of different solutions to this problem have been devised in recent years, each of which is described below:

1.2.1  Paravirtualization

Under paravirtualization, the kernel of the guest operating system is modified specifically to run on the hypervisor. This typically involves replacing any privileged operations that will only run in ring 0 of the CPU with calls to the hypervisor (known as hypercalls). The hypervisor, in turn, performs the task on behalf of the guest kernel. This typically limits support to open source operating systems such as Linux which may be freely altered and proprietary operating An Overview of Virtualization Techniques

systems where the owners have agreed to make the necessary code modifications to target a specific hypervisor. These issues notwithstanding, the ability of the guest kernel to communicate directly with the hypervisor results in greater performance levels compared to other virtualization approaches.

1.2.2  Full Virtualization

Full virtualization provides support for unmodified guest operating systems. The term unmodified refers to operating system kernels which have not been altered to run on a hypervisor and therefore still execute privileged operations as though running in ring 0 of the CPU. In this scenario, the hypervisor provides CPU emulation to handle and modify privileged and protected CPU operations made by unmodified guest operating system kernels. Unfortunately this emulation process requires both time and system resources to operate resulting in inferior performance levels when compared to those provided by paravirtualization.

1.2.3  Hardware Virtualization

Hardware virtualization leverages virtualization features built into the latest generations of CPUs from both Intel and AMD. These technologies, known as Intel VT and AMD-V respectively, provide extensions necessary to run unmodified guest virtual machines without the overheads inherent in full virtualization CPU emulation. In very simplistic terms these processors provide an additional privilege mode (referred to as ring -1) above ring 0 in which the hypervisor can operate, thereby leaving ring 0 available for unmodified guest operating systems. The following figure illustrates the Type-1 hypervisor approach to virtualization:

Figure 24-3

As outlined in the above illustration, in addition to the virtual machines, an administrative operating system and/or management console also runs on top of the hypervisor allowing the virtual machines to be managed by a system administrator.

1.3  Virtual Machine Networking

Virtual machines will invariably need to be connected to a network to be of any practical use. One option is for the guest to be connected to a virtual network running within the operating system of the host computer. In this configuration any virtual machines on the virtual network can see each other but access to the external network is provided by Network Address Translation (NAT). When using the virtual network and NAT, each virtual machine is represented on the external network (the network to which the host is connected) using the IP address of the host system. This is the default behavior for KVM virtualization on Ubuntu and generally requires no additional configuration. Typically, a single virtual network is created by default, represented by the name default and the device virbr0.

In order for guests to appear as individual and independent systems on the external network (i.e. with their own IP addresses), they must be configured to share a physical network interface on the host. The quickest way to achieve this is to configure the virtual machine to use the “direct connection” network configuration option (also referred to a MacVTap) which will provide the guest system with an IP address on the same network as the host. Unfortunately, while this gives the virtual machine access to other systems on the network, it is not possible to establish a connection between the guest and the host when using the MacVTap driver.

A better option is to configure a network bridge interface on the host system to which the guests can connect. This provides the guest with an IP address on the external network while also allowing the guest and host to communicate, a topic which is covered in the chapter entitled “Creating an Ubuntu KVM Networked Bridge Interface”.

1.4  Summary

Virtualization is defined as the ability to run multiple guest operating systems within a single host operating system. A number of approaches to virtualization have been developed including guest operating system and hypervisor virtualization. Hypervisor virtualization falls into two categories known as Type-1 and Type-2. Type-2 virtualization solutions are categorized as paravirtualization, full virtualization and hardware virtualization, the latter making use of special virtualization features of some Intel and AMD processor models.

Virtual machine guest operating systems have a number of options in terms of networking including NAT, direct connection (MacVTap) and network bridge configurations.

Sharing Files between Ubuntu 20.04 and Windows Systems with Samba

Although Linux has made some inroads into the desktop market, its origins and future are very much server-based. It is not surprising therefore that Ubuntu has the ability to act as a file server. It is also extremely common for Ubuntu and Windows systems to be used side by side in networked environments. It is a common requirement, therefore, that files on an Ubuntu system be accessible to Linux, UNIX and Windows-based systems over network connections. Similarly, shared folders and printers residing on Windows systems may also need to be accessible from Ubuntu based systems.

Windows systems share resources such as file systems and printers using a protocol known as Server Message Block (SMB). In order for an Ubuntu system to serve such resources over a network to a Windows system and vice versa it must, therefore, support SMB. This is achieved using technology called Samba. In addition to providing integration between Linux and Windows systems, Samba may also be used to provide folder sharing between Linux systems (as an alternative to NFS which was covered in the previous chapter).

In this chapter we will look at the steps necessary to share file system resources and printers on an Ubuntu system with remote Windows and Linux systems, and to access Windows resources from Ubuntu.

1.1  Accessing Windows Resources from the GNOME Desktop

Before getting into more details of Samba sharing, it is worth noting that if all you want to do is access Windows shared folders from within the Ubuntu GNOME desktop then support is already provided within the GNOME Files application. The Files application is located in the dash as highlighted in Figure 23-1:

Figure 23-1

Once launched, select the Other Locations option in the left-hand navigation panel followed by the Windows Network icon in the main panel to browse available windows resources:

Figure 23-2

1.2  Samba and Samba Client

Samba allows both Ubuntu resources to be shared with Windows systems and Windows resources to be shared with Ubuntu systems. Ubuntu accesses Windows resources using the Samba client. Ubuntu resources, on the other hand, are shared with Windows systems by installing and configuring the Samba service.

1.3  Installing Samba on an Ubuntu System

The default settings used during the Ubuntu installation process do not typically install the necessary Samba packages. Unless you specifically requested that Samba be installed it is unlikely that you have Samba installed on your system. To check whether Samba is installed, open a terminal window and run the following command:

# apt -qq list samba-common samba smbclient

Any missing packages can be installed using the apt command-line tool:

# apt install samba-common samba smbclient

1.4  Configuring the Ubuntu Firewall to Enable Samba

Next, the firewall currently protecting the Ubuntu system needs to be configured to allow Samba traffic.

If you are using the Uncomplicated Firewall (ufw) run the following command:

# ufw allow samba

Alternatively, if you are using firewalld, run the firewall-cmd command as follows:

# firewall-cmd --permanent --add-port={139/tcp,445/tcp}
# firewall-cmd --reload

Before starting the Samba service a number of configuration steps are necessary to define how the Ubuntu system will appear to Windows systems, and the resources which are to be shared with remote clients. The majority of these configuration tasks take place within the /etc/samba/smb. conf file.

1.5  Configuring the smb.conf File

Samba is a highly flexible and configurable system that provides many different options for controlling how resources are shared on Windows networks. This flexibility can lead to the sense that Samba is overly complex to work with. In reality, however, many of the configuration options are not needed by the typical installation, and the learning curve to set up a basic configuration is actually quite short.

For the purposes of this chapter we will look at joining an Ubuntu system to a Windows workgroup and setting up a directory as a shared resource that can be accessed by a specific user. This is a configuration known as a standalone Samba server. More advanced configurations such as integrating Samba within an Active Directory environment are also available, though these are outside the scope of this book.

The first step in configuring Samba is to edit the /etc/samba/smb.conf file.

1.5.1  Configuring the [global] Section

The smb.conf file is divided into sections. The first section is the [global] section where settings can be specified that apply to the entire Samba configuration. While these settings are global, each option may be overridden within other sections of the configuration file.

The first task is to define the name of the Windows workgroup on which the Ubuntu resources are to be shared. This is controlled via the workgroup = directive of the [global] section which by default is configured as follows:

workgroup = WORKGROUP

Begin by changing this to the actual name of the workgroup if necessary.

In addition to the workgroup setting, the other settings indicate that this is a standalone server on which the shared resources will be protected by user passwords. Before moving on to configuring the resources to be shared, other parameters also need to be added to the [global] section as follows:

[global]
.
.
        netbios name = LinuxServer
.
.

The “netbios name” property specifies the name by which the server will be visible to other systems on the network.

1.5.2  Configuring a Shared Resource

The next step is to configure the shared resources (in other words the resources that will be accessible from other systems on the Windows network). In order to achieve this, the section is given a name by which it will be referred to when shared. For example, if we plan to share the /sampleshare directory of our Ubuntu system, we might entitle the section [sampleshare]. In this section a variety of configuration options are possible. For the purposes of this example, however, we will simply define the directory that is to be shared, indicate that the directory is both browsable and writable and declare the resource public so that guest users are able to gain access:

[sampleshare]
        comment = Example Samba share
        path = /sampleshare
        browseable = Yes
        public = yes
        writable = yes

To restrict access to specific users, the “valid users” property may be used, for example:

valid users = demo, bobyoung, marcewing

1.5.3  Removing Unnecessary Shares

The smb.conf file is pre-configured with sections for sharing printers and the home folders of the users on the system. If these resources do not need to be shared, the corresponding sections can be commented out so that they are ignored by Samba. In the following example, the [homes] section has been commented out:

.
.
#[homes]
#       comment = Home Directories
#       valid users = %S, %D%w%S
#       browseable = No
#       read only = No
#       inherit acls = Yes
.
.

1.6  Creating a Samba User

Any user that requires access to a Samba shared resource must be configured as a Samba User and assigned a password. This task is achieved using the smbpasswd command-line tool. Consider, for example, that a user named demo is required to be able to access the /sampleshare directory of our Ubuntu system from a Windows system. In order to fulfill this requirement we must add demo as a Samba user as follows:

# smbpasswd -a demo
New SMB password:
Retype new SMB password:
Added user demo.

Now that we have completed the configuration of a very basic Samba server, it is time to test our configuration file and then start the Samba services.

1.7  Testing the smb.conf File

The settings in the smb.conf file may be checked for errors using the testparm command-line tool as follows:

# testparm
Load smb config files from /etc/samba/smb.conf
rlimit_max: increasing rlimit_max (1024) to minimum Windows limit (16384)
WARNING: The "syslog" option is deprecated
Processing section "[printers]"
Processing section "[print$]"
Processing section "[sampleshare]"
Loaded services file OK.
Server role: ROLE_STANDALONE
 
Press enter to see a dump of your service definitions
 
# Global parameters
[global]
	dns proxy = No
	log file = /var/log/samba/log.%m
	map to guest = Bad User
	max log size = 1000
	netbios name = LINUXSERVER
	obey pam restrictions = Yes
	pam password change = Yes
	panic action = /usr/share/samba/panic-action %d
	passwd chat = *Enter\snew\s*\spassword:* %n\n *Retype\snew\s*\spassword:* %n\n *password\supdated\ssuccessfully* .
	passwd program = /usr/bin/passwd %u
	security = USER
	server role = standalone server
	server string = %h server (Samba, Ubuntu)
	syslog = 0
	unix password sync = Yes
	usershare allow guests = Yes
	wins support = Yes
	idmap config * : backend = tdb
 
[printers]
	browseable = No
	comment = All Printers
	create mask = 0700
	path = /var/spool/samba
	printable = Yes
 
[print$]
	comment = Printer Drivers
	path = /var/lib/samba/printers
 
[sampleshare]
	comment = Example Samba share
	guest ok = Yes
	path = /sampleshare
	read only = No

1.8  Starting the Samba and NetBIOS Name Services

In order for an Ubuntu server to operate within a Windows network both the Samba (SMB) and NetBIOS nameservice (NMB) services must be started. Optionally, also enable the services so that they start each time the system boots:

# systemctl enable smbd
# systemctl start smbd
# systemctl enable nmbd
# systemctl start nmbd

Before attempting to connect from a Windows system, use the smbclient utility to verify that the share is configured:

# smbclient -U demo -L localhost 
Enter WORKGROUP\demo's password: 
 
	Sharename       Type      Comment
	---------       ----      -------
	print$          Disk      Printer Drivers
	sampleshare     Disk      Example Samba share
	IPC$            IPC       IPC Service (demo-server2 server (Samba, Ubuntu))
	Officejet_Pro_8600_C7C718_ Printer   
	Officejet_6600_971B9B_ Printer   
Reconnecting with SMB1 for workgroup listing.
 
	Server               Comment
	---------            -------
 
	Workgroup            Master
	---------            -------
	WORKGROUP            LINUXSERVER

1.9  Accessing Samba Shares

Now that the Samba resources are configured and the services are running, it is time to access the shared resource from a Windows system. On a suitable Windows system on the same workgroup as the Ubuntu system, open Windows Explorer and navigate to the Network panel. At this point, explorer should search the network and list any systems using the SMB protocol that it finds. The following figure illustrates an Ubuntu system named LINUXSERVER located using Windows Explorer on a Windows 10 system:

Figure 23-3

Double clicking on the LINUXSERVER host will prompt for the name and password of a user with access privileges. In this case it is the demo account that we configured using the smbpasswd tool:

Figure 23-4

Entering the username and password will result in the shared resources configured for that user appearing in the explorer window, including the previously configured /sampleshare resource:

Figure 23-5

Double clicking on the /sampleshare shared resource will display a listing of the files and directories contained therein.

If you are unable to see the Linux system or have problems accessing the shared folder, try mapping the Samba share to a local Windows drive as follows:

  1. Open Windows File Explorer, right-click on the Network entry in the left-hand panel and select Map network drive… from the resulting menu.
  2. From the Map Network Drive dialog, select a drive letter before entering the path to the shared folder. For example:
\\LinuxServer\sampleshare

Enable the checkbox next to Connect using different credentials. If you do not want the drive to be mapped each time you log into the Windows system, turn off the corresponding check box:

Figure 23-6

With the settings entered, click on the Finish button to map the drive, entering the username and password for the Samba user configured earlier in the chapter when prompted. After a short delay the content of the Samba share will appear in a new File Explorer window.

1.10  Accessing Windows Shares from Ubuntu

As previously mentioned, Samba is a two way street, allowing not only Windows systems to access files and printers hosted on an Ubuntu system, but also allowing the Ubuntu system to access shared resources on Windows systems. This is achieved using the smbclient package which was installed at the start of this chapter. If it is not currently installed, install it from a terminal window as follows:

# apt install smbclient

Shared resources on a Windows system can be accessed either from the Ubuntu desktop using the Files application, or from the command-line prompt using the smbclient and mount tools. The steps in this section assume that appropriate network sharing settings have been enabled on the Windows system.

To access any shared resources on a Windows system using the GNOME desktop, begin by launching the Files application and selecting the Other Locations option. This will display the screen shown in Figure 23-7 below including an icon for the Windows Network (if one is detected):

Figure 23-7

Selecting the Windows Network option will display the Windows systems detected on the network and allow access to any shared resources.

Figure 23-8

Alternatively, the Connect to Server option may be used to connect to a specific system. Note that the name or IP address of the remote system must be prefixed by smb:// and may be followed by the path to a specific shared resource, for example:

smb://WinServer10/Documents

1.11  Summary

In this chapter we have looked at how to configure an Ubuntu system to act as both a Samba client and server allowing the sharing of resources with Windows systems. Topics covered included the installation of Samba client and server packages and configuration of Samba as a standalone server.

Using NFS to Share Ubuntu 20.04 Files with Remote Systems

Ubuntu provides two mechanisms for sharing files and folders with other systems on a network. One approach is to use technology called Samba. Samba is based on Microsoft Windows Folder Sharing and allows Linux systems to make folders accessible to Windows systems, and also to access Windows based folder shares from Linux. This approach can also be used to share folders between other Linux and UNIX based systems as long as they too have Samba support installed and configured. This is by far the most popular approach to sharing folders in heterogeneous network environments. The topic of folder sharing using Samba is covered in “Sharing Files between Ubuntu and Windows Systems with Samba”.

Another option, which is targeted specifically at sharing folders between Linux and UNIX based systems, uses technology called Network File System (NFS). NFS allows the file system on one Linux computer to be accessed over a network connection by another Linux or UNIX system. NFS was originally developed by Sun Microsystems (now part of Oracle Corporation) in the 1980s and remains the standard mechanism for sharing of remote Linux/UNIX file systems to this day.

NFS is very different to the Windows SMB resource sharing technology used by Samba. In this chapter we will be looking at network based sharing of folders between Ubuntu and other UNIX/ Linux based systems using NFS.

1.1  Ensuring NFS Services are running on Ubuntu

The first task is to verify that the NFS services are installed and running on your Ubuntu system. This can be achieved either from the command-line, or using the Cockpit interface.

Begin by installing the NFS service by running the following command from a terminal window:

# apt install nfs-kernel-server

Next, configure the service to automatically start at boot time:

# systemctl enable nfs-kernel-server

Once the service has been enabled, start it as follows:

# systemctl start nfs-kernel-server

1.2  Configuring the Ubuntu Firewall to Allow NFS Traffic

Next, the firewall needs to be configured to allow NFS traffic.

If the Uncomplicated Firewall is enabled, run the following command to add a rule to allow NFS traffic:

# ufw allow nfs

If, on the other hand, you are using firewalld, run the following firewall-cmd commands where <zone> is replaced by the appropriate zone for your firewall and system configuration:

# firewall-cmd --zone=<zone> --permanent --add-service=mountd
# firewall-cmd --zone=<zone> --permanent --add-service=nfs
# firewall-cmd --zone=<zone> --permanent --add-service=rpc-bind
# firewall-cmd --reload

1.3  Specifying the Folders to be Shared

Now that NFS is running and the firewall has been configured, we need to specify which parts of the Ubuntu file system may be accessed by remote Linux or UNIX systems. These settings can be declared in the /etc/exports file, which will need to be modified to export the directories for remote access via NFS. The syntax for an export line in this file is as follows:

<export> <host1>(<options>) <host2>(<options>)...

In the above line, <export> is replaced by the directory to be exported, <host1> is the name or IP address of the system to which access is being granted and <options> represents the restrictions that are to be imposed on that access (read only, read write etc). Multiple host and options entries may be placed on the same line if required. For example, the following line grants read only permission to the /datafiles directory to a host with the IP address of 192.168.2.38:

/datafiles 192.168.2.38(ro,no_subtree_check)

The use of wildcards is permitted in order to apply an export to multiple hosts. For example, the following line permits read write access to /home/demo to all external hosts:

/home/demo *(rw)

A full list of options supported by the exports file may be found by reading the exports man page:

# man exports

For the purposes of this chapter, we will configure the /etc/exports file as follows:

/tmp       *(rw,sync,no_subtree_check)
/vol1      192.168.2.21(ro,sync,no_subtree_check)

Once configured, the table of exported file systems maintained by the NFS server needs to be updated with the latest /etc/exports settings using the exportfs command as follows:

# exportfs -a

It is also possible to view the current share settings from the command-line using the exportfs tool:

# exportfs

The above command will generate the following output:

/tmp            <world>
/vol1           192.168.2.21

Using NFS to Share Ubuntu Files with Remote Systems

1.4  Accessing Shared Ubuntu Folders

The shared folders may be accessed from a client system by mounting them manually from the command-line. Before attempting to mount a remote NFS folder, the nfs-common package should first be installed on the client system:

# apt install nfs-common

To mount a remote folder from the command-line, open a terminal window and create a directory where you would like the remote shared folder to be mounted:

# mkdir /home/demo/tmp

Next enter the command to mount the remote folder using either the IP address or hostname of the remote NFS server, for example:

# mount -t nfs 192.168.1.115:/tmp /home/demo/tmp

The remote /tmp folder will then be mounted on the local system. Once mounted, the /home/ demo/tmp folder will contain the remote folder and all its contents.

Options may also be specified when mounting a remote NFS filesystem. The following command, for example, mounts the same folder, but configures it to be read-only:

# mount -t nfs -o ro 192.168.1.115:/tmp /home/demo/tmp

1.5  Mounting an NFS Filesystem on System Startup

It is also possible to configure an Ubuntu system to automatically mount a remote file system each time the system starts up by editing the /etc/fstab file. When loaded into an editor, it will likely resemble the following:

UUID=84982a2e-0dc1-4612-9ffa-13baf91ec558 /     ext4    errors=remount-ro 0  1
/swapfile                        none            swap    sw              0       0

To mount, for example, a folder with the path /tmp which resides on a system with the IP address 192.168.1.115 in the local folder with the path /home/demo/tmp (note that this folder must already exist) add the following line to the /etc/fstab file:

192.168.1.115:/tmp      /home/demo/tmp           nfs     rw              0 0

Next time the system reboots the /tmp folder located on the remote system will be mounted on the local /home/demo/tmp mount point. All the files in the remote folder can then be accessed as if they reside on the local hard disk drive.

1.6  Unmounting an NFS Mount Point

Once a remote file system is mounted using NFS it can be unmounted using the umount command with the local mount point as the command-line argument. The following command, for example, will unmount our example filesystem mount point:

# umount /home/demo/tmp

1.7  Accessing NFS Filesystems in Cockpit

In addition to mounting a remote NFS file system on a client using the command-line, it is also possible to perform mount operations from within the Cockpit web interface. Assuming that Cockpit has been installed and configured on the client system, log into the Cockpit interface from within a web browser and select the Storage option from the left-hand navigation panel. If the Storage option is not listed, the cockpit-storaged package will need to be installed:

# apt install cockpit-storaged

Once the Cockpit service has restarted, log back into the Cockpit interface at which point the Storage option should now be visible.

Once selected, the main storage page will include a section listing any currently mounted NFS file systems as illustrated in Figure 22-1:

Figure 22-1

To mount a remote filesystem, click on the ‘+’ button highlighted above and enter information about the remote NFS server and file system share together with the local mount point and any necessary options into the resulting dialog before clicking on the Add button:

Figure 22-2

To modify, unmount or remove an NFS filesystem share, select the corresponding mount in the NFS Mounts list (Figure 22-1 above) to display the page shown in Figure 22-3 below:

Using NFS to Share Ubuntu Files with Remote Systems

Figure 22-3

1.8  Summary

The Network File System (NFS) is a client/server-based system, originally developed by Sun Microsystems, which provides a way for Linux and Unix systems to share filesystems over a network. NFS allows a client system to access and (subject to permissions) modify files located on a remote server as though those files are stored on a local filesystem. This chapter has provided an overview of NFS and outlined the options available for configuring both client and server systems using the command-line or the Cockpit web interface.

Displaying Ubuntu 20.04 Applications Remotely (X11 Forwarding)

In the previous chapter we looked at how to display the entire Ubuntu desktop on a remote computer. While this works well if you actually need to remotely display the entire desktop, it could be considered overkill if all you want to do is display a single application. In this chapter, therefore, we will look at displaying individual applications on a remote system.

1.1  Requirements for Remotely Displaying Ubuntu Applications

In order to run an application on one Ubuntu system and have it display on another system there are a couple of prerequisites. First, the system on which the application is to be displayed must be running an X server. If the system is a Linux or UNIX-based system with a desktop environment running then this is no problem. If the system is running Windows or macOS, however, then you must install an X server on it before you can display applications from a remote system. A number of commercial and free Windows based X servers are available for this purpose and a web search should provide you with a list of options.

Second, the system on which the application is being run (as opposed to the system on which the application is to be displayed) must be configured to allow SSH access. Details on configuring SSH on an Ubuntu system can be found in the chapter entitled “Configuring SSH Key-based Authentication on Ubuntu”. This system must also be running the X Window system from X.org instead of Wayland. To find out which system is being used, open a terminal window and run the following command:

# echo $XDG_SESSION_TYPE x11

If the above command outputs “wayland” instead of “x11”, edit the /etc/gdm3/custom.conf file and uncomment the WaylandEnable line as follows and restart the system:

# Uncomment the line below to force the login screen to use Xorg
WaylandEnable=false

Finally, SSH must be configured to allow X11 forwarding. This is achieved by adding the following directive to the SSH configuration on the system from which forwarding is to occur. Edit the /etc/ ssh/ssh_config file and uncomment the ForwardX11 entry (in other words remove the ‘#’ at the beginning of the line) and change the value to yes entry as follows:

.
.
Host *
#   ForwardAgent no
    ForwardX11 yes
.
.

After making the change, save the file and restart the SSH service:

# systemctl restart sshd

Once the above requirements are met it should be possible to remotely display an X-based desktop application.

1.2  Remotely Displaying an Ubuntu Application

The first step in remotely displaying an application is to move to the system where the application is to be displayed. At this system, establish an SSH connection to the remote system so that you have a command prompt. This can be achieved using the ssh command. When using the ssh command we need to use the -X flag to tell it that we plan to tunnel X11 traffic through the connection:

$ ssh -X user@hostname

In the above example user is the user name to use to log into the remote system and hostname is the hostname or IP address of the remote system. Enter your password at the login prompt and, once logged in, run the following command to see the DISPLAY setting:

$ echo $DISPLAY

The command should output something similar to the following: localhost:10.0

To display an application simply run it from the command prompt. For example:

$ gedit

When executed, the above command should run the gedit tool on the remote system, but display the user interface on the local system.

1.3  Trusted X11 Forwarding

If the /etc/ssh/ssh_config file on the remote system contains the following line, then it is possible to use trusted X11 forwarding:

ForwardX11Trusted yes

Trusted X11 forwarding is slightly faster than untrusted forwarding but is less secure since it does not engage the X11 security controls. The -Y flag is needed when using trusted X11 forwarding:

$ ssh -Y user@hostname

1.4  Compressed X11 Forwarding

When using slower connections the X11 data can be compressed using the -C flag to improve performance:

$ ssh -X -C user@hostname

1.5  Displaying Remote Ubuntu Apps on Windows

To display Ubuntu based apps on Windows an SSH client and an X server will need to be installed on the Windows system. The subject of installing and using the PuTTY client on Windows was covered earlier in the book in the “Configuring SSH Key-based Authentication on Ubuntu” chapter. Refer to this chapter if you have not already installed PuTTY on your Windows system.

In terms of the X server, a number of options are available, though a popular choice appears to be VcXsrv which is available for free from the following URL:

https://sourceforge.net/projects/vcxsrv/

Once the VcXsrv X server has been installed, an application named XLaunch will appear on the desktop and in the start menu. Start XLaunch and select a display option (the most flexible being the Multiple windows option which allows each client app to appear in its own window):

Figure 21-1

Click the Next button to proceed through the remaining screens, accepting the default configuration settings. On the final screen, click on the Finish button to start the X server. If the Windows Defender dialog appears click on the button to allow access to your chosen networks.

Once running, XLaunch will appear in the taskbar and can be exited by right-clicking on the icon and selecting the Exit… menu option:

Figure 21-2

With the X server installed and running, launch PuTTY and either enter the connection information for the remote host or load a previously saved session profile. Before establishing the connection, however, X11 forwarding needs to be enabled. Within the PuTTY main window, scroll down the options in the left-hand panel, unfold the SSH section and select the X11 option as shown in Figure 21-3:

Figure 21-3

Turn on the Enable X11 forwarding checkbox highlighted in Figure 21-4, return to the sessions screen and open the connection (saving the session beforehand if you plan to use it again).

Displaying Ubuntu Applications Remotely (X11 Forwarding)

Figure 21-4

Log into the Ubuntu system within the PuTTY session window and run a desktop app. After a short delay, the app will appear in the Windows desktop in its own window. Any dialogs that are opened by the app will also appear in separate windows, just as they would on the Ubuntu GNOME desktop. Figure 21-5, for example, shows the Ubuntu nm-connection-editor tool displayed on a Windows 10 system:

Figure 21-5

1.6  Summary

For situations where remote access to individual Ubuntu desktop applications is required as opposed to the entire GNOME desktop, X11 forwarding provides a lightweight solution to remotely displaying graphical applications. The system on which the applications are to appear must be running an X Window System based desktop environment (such as GNOME) or have an X server installed and running. Once X11 forwarding has been enabled on the remote server and a secure SSH connection established from the local system using the X11 forwarding option, most applications can be displayed remotely on the local X server.

Ubuntu 20.04 Remote Desktop Access with VNC

The chapter entitled “Ubuntu Remote Desktop Access with Vino” explored remote access to the Ubuntu GNOME desktop using the Vino server, an approach that is intended solely for situations where the remote system is already running a GNOME desktop session. In this chapter we will cover launching and accessing GNOME desktop sessions that run in the background, allowing multiple desktop sessions to be accessed remotely, including on server based system that do not have a graphical console attached.

1.1  Installing the GNOME Desktop Environment

It is, of course, only possible to access the desktop environment if the desktop itself has been installed. If, for example, the system was initially configured as a server it is unlikely that the desktop packages were installed. The easiest way to install the packages necessary to run the GNOME desktop is via the apt command as follows:

# apt install ubuntu-gnome-desktop

To prevent the desktop from attempting to launch automatically each time the system reboots, change the default systemd target back to multi-user:

# systemctl set-default multi-user.target

If the system has a graphical display attached, the desktop can be launched using the following command:

$ startx

If, on the other hand, the system is a server with no directly connected display, the only way to run and access the desktop will be to configure VNC support on the system.

1.2  Installing VNC on Ubuntu

Access to a remote desktop requires a VNC server installed on the remote system, a VNC viewer on the system from which access is being established and, optionally, a secure SSH connection. While a number of VNC server and viewer implementations are available, this chapter will make use of TigerVNC which provides both server and viewer components for Linux-based operating systems. VNC viewer clients for non-Linux platforms include RealVNC and TightVNC.

To install the TigerVNC server package on Ubuntu, simply run the following command:

# apt install tigervnc-standalone-server

If required, the TigerVNC viewer may also be installed as follows:

# apt install tigervnc-viewer

Once the server has been installed the system will need to be configured to run one or more VNC services and to open the appropriate ports on the firewall.

1.3  Configuring the VNC Server

With the VNC server packages installed, the next step is to configure the server. The first step is to specify a password for the user that will be accessing the remote desktop environment. While logged in as root (or with superuser privileges), execute the vncpasswd command (where the user name is assumed to be demo):

# su - demo
demo@demoserver:~$ vncpasswd
Password:
Verify:
Would you like to enter a view-only password (y/n)? n
A view-only password is not used

The above command will create a file named passwd in the .vnc directory of the user’s home directory. Next, change directory to the .vnc directory and create a new file named xstartup containing the following:

#!/bin/sh
# Start Gnome 3 Desktop 
[ -x /etc/vnc/xstartup ] && exec /etc/vnc/xstartup
[ -r $HOME/.Xresources ] && xrdb $HOME/.Xresources
vncconfig -iconic &
dbus-launch --exit-with-session gnome-session &

These are the commands that will be executed to start the GNOME desktop when the VNC server is launched.

1.4  Starting the VNC Server

With the necessary packages installed and configured for the user’s account, the VNC server can be started as follows (making sure to run the command as the user and without superuser privileges):

$ vncserver

This will start the first desktop session running on the system. Since this is the first session, it will be configured to use port 5901 (which may be abbreviated to :1). Running the command a second time while the first session is running will create a VNC server listening on port 5902 (:2) and so on. The following command may be used to obtain a list of desktop sessions currently running:

$ vncserver -list
TigerVNC server sessions:

X DISPLAY #	PROCESS ID
:1		1607
:2		4726

To terminate a session, use the vncserver command with the -kill option referencing the corresponding port. For example:

$ vncserver -kill :2
Killing Xtigervnc process ID 4726... success!

Alternatively, use the following command to kill all currently running VNC server sessions:

$ vncserver -kill :*
Killing Xtigervnc process ID 1607... success!
Killing Xtigervnc process ID 5287... success!

To manually specify the port to be used by the VNC server session, include the number in the command-line as follows:

$ vncserver :5

In the above example, the session will listen for a remote connection on port 5905.

1.5  Connecting to a VNC Server

For details on remotely connecting to a desktop session from another system, follow the steps outlined in the sections titled “Establishing a Secure Remote Desktop Session” and “Establishing a Secure Tunnel on Windows using PuTTY” in the previous chapter.

1.6  Summary

In this and the preceding chapter we have explored two different ways to remotely access the GNOME desktop environment of an Ubuntu system. While the previous chapter explored access to an existing desktop session, this chapter has focused on launching GNOME desktop sessions as background processes, thereby allowing remote access to multiple desktop sessions. This is a particularly useful technique for running and remotely accessing desktop sessions on “headless” server-based systems.

Ubuntu 20.04 Remote Desktop Access with Vino

Ubuntu can be configured to provide remote access to the graphical desktop environment over a network or internet connection. Although not enabled by default, it is relatively straightforward to display and access an Ubuntu desktop from a system anywhere else on a network or the internet. This can be achieved regardless of whether that system is running Linux, Windows or macOS. In fact, there are even apps available for Android and iOS that will allow you to access your Ubuntu desktop from just about anywhere that a data signal is available.

Remote desktop access can be useful in a number of scenarios. It enables you or another person, for example, to view and interact with your Ubuntu desktop environment from another computer system either on the same network or over the internet. This is useful if you need to work on your computer when you are away from your desk such as while traveling. It is also useful in situations where a co-worker or IT support technician needs access to your desktop to resolve a problem.

The Ubuntu remote desktop functionality is based on technology known as Virtual Network Computing (VNC) and in this and the next chapter we will cover the key aspects of configuring and using remote desktops within Ubuntu.

1.1  Remote Desktop Access Types

Before starting it is important to understand that there are essentially two types of remote desktop access. The approach covered in this chapter is useful if you primarily use Ubuntu as a desktop operating system and require remote access to your usual desktop session. When configured, you will take over your desktop session and view and control it remotely.

The second option, covered in the next chapter entitled “Ubuntu Remote Desktop Access with VNC”, is intended for situations where you need to start and access one or more remote desktop sessions on a remote server-based system, regardless of whether the remote system has a graphical console attached. This allows you to launch multiple desktop sessions in the background on the remote system and view and control those desktops over a network or internet connection.

1.2  Secure and Insecure Remote Desktop Access

In this chapter we will cover both secure and insecure remote desktop access methods. Assuming that you are accessing one system from another within the context of a secure internal network then it is generally safe to use the insecure access method. If, on the other hand, you plan to access your desktop remotely over any kind of public network you must use the secure method of access to avoid your system and data being compromised.

Remote desktop access on Ubuntu is provided by the Vino package. Vino is a VNC server that was developed specifically for use with the GNOME desktop.

The first step in enabling remote access is to install this package:

# apt install vino

Once Vino has been installed, the next step is to enable remote desktop access from within GNOME. Begin by opening the settings app as shown in Figure 19-1:

Figure 19-1

From within the Settings application, select the Sharing option (marked A in Figure 19-2):

Figure 19-2

Turn on the Sharing switch (B) and click on the Screen Sharing option (C) to display the dialog shown in Figure 19-3 below:

Figure 19-3

The Screen Sharing dialog provides the following configuration options to manage remote desktop access:

  • Allows connections to control the screen – If enabled, the remote session will be able to use the mouse and keyboard to interact with the desktop environment. If disabled the remote session will only allow the desktop to be viewed.
  • New connections must ask for access – When selected, a prompt will appear on the host screen asking to give permission to the remote user to access the desktop. Do not select this option if you plan to access your screen remotely and nobody will be at the host system to accept the connection request.
  • Require a password – Requires the user to enter the specified password prior to gaining access to the desktop.
  • Networks – The network connections on the host system via which remote access is to be permitted. After configuring the settings, close both the Screen Settings and Settings dialogs.

1.4  Connecting to the Shared Desktop

Although VNC viewer implementations are available for a wide range of operating systems, a tool such as the Remmina Desktop Client is recommended when connecting from Ubuntu or other Linux-based systems. Remmina is a user friendly tool with a graphical interface that supports the encryption used by Vino to ensure a secure remote connection.

To install this tool, open the Ubuntu Software application and search for and install Remmina:

Figure 19-4

After installing and launching Remmina, change the connection type menu (marked A in Figure 19-5) to VNC and enter into the address field (B) the IP address or hostname of the remote system to which you wish to connect:

Figure 19-5

To establish the connection, tap the keyboard Enter key to begin the connection process. After a short delay, a second screen will appear requesting the desktop access password (if one was entered when screen sharing was enabled earlier in the chapter):

Figure 19-6

After entering the password, click on OK to access the remote screen:

Figure 19-7

The default settings for Remmina prioritize speed over image quality. If you find that the quality of the desktop rendering is unacceptably poor, click on the settings button (Figure 19-8) in the left-hand toolbar within the remote viewer window and experiment with different settings until you find the ideal balance of performance and image quality:

Figure 19-8

1.5  Connecting from Non-Linux Clients

The previous section assumed that the remote desktop was being accessed from a Linux or UNIX system. It is important to understand that Vino, by default, requires that the remote connection be encrypted to ensure security. One of the reasons for using Remmina is that it fully supports the encryption used by Vino.

If you need to connect from a non-Linux system such as Windows or macOS you will need to install a third-party VNC viewer such as TightVNC, TigerVNC or RealVNC. Unfortunately, these viewers do not support the encryption that Vino uses by default. To experience this in action, download the RealVNC viewer for your macOS or Windows system from the following URL:

https://www.realvnc.com/en/connect/download/viewer/

Once installed, launch the viewer and enter the hostname or IP address of your remote Ubuntu system. On attempting to connect, a failure dialog will appear similar to the one shown in Figure 19-9:

Figure 19-9

To allow a connection to be established from a non-Linux system it is necessary to turn off the encryption requirement for the remote desktop connection. To do this, open a terminal window on the Ubuntu system and run the following command (using your account and without sudo privileges):

$ gsettings set org.gnome.Vino require-encryption false

After disabling the Vino encryption requirement, attempt to connect from the RealVNC viewer once again. This time a warning will appear indicating that the connection is not encrypted:

Figure 19-10

Clicking the Continue button will dismiss the warning dialog and establish the remote connection.

Clearly, connecting to a remote VNC server using the steps in this section results in an insecure, unencrypted connection between the client and server. This means that the data transmitted during the remote session is vulnerable to interception. To establish a secure and encrypted connection from an Ubuntu system to a non-Linux client a few extra steps are necessary.

1.6  Establishing a Secure Remote Desktop Session

The remote desktop connection from macOS and Windows in the previous section is considered to be insecure because no encryption is used. This is acceptable when the remote connection does not extend outside of an internal network protected by a firewall. When a remote session is required over an internet connection, however, a more secure option is needed. This is achieved by tunneling the remote desktop through a secure shell (SSH) connection. This section will cover how to do this on Linux, UNIX and macOS client systems.

When a remote desktop session is invoked on an Ubuntu system a connection is made using TCP/IP network port 5900. To prove this, establish a connection to your remote Ubuntu system referencing port 5900 after the hostname or IP address, for example, and note that the connection is still established:

192.168.86.218:5900

To implement an encrypted remote desktop session for non-Linux system the session needs to be tunneled through a secure SSH connection.

If the SSH server has not yet been installed on your Ubuntu system, refer to the chapter entitled “Configuring SSH Key-based Authentication on Ubuntu”.

Assuming the SSH server is installed and active it is time to move to the other system. At the other system, log in to the remote system using the following command, which will establish the secure tunnel between the two systems. This assumes the client system is running macOS, Linux or UNIX (instructions for Windows systems are covered in the next section):

$ ssh -l <username> -L 5900:localhost:5900 <remotehost>

In the above example, <username> references the user account on the remote system for which VNC access has been configured, and <remotehost> is either the host name or IP address of the remote system, for example:

$ ssh -l demo -L 5900:localhost:5900 192.168.86.218

When prompted, log in using the account password. With the secure connection established it is time to launch vncviewer so that it uses the secure tunnel. Leaving the SSH session running in the terminal window, launch the VNC viewer and enter the following into the address field: localhost:5900

The vncviewer session will prompt for a password if one is required, and then launch the VNC viewer providing secure access to your desktop environment.

Although the connection is now secure and encrypted, the VNC viewer will most likely still report that the connection is insecure. Unfortunately, although the connection is now secure, the VNC viewer software has no way of knowing this and consequently continues to issue warnings. Rest assured that as long as the SSH tunnel is being used, the connection is indeed secure.

In the above example we left the SSH tunnel session running in a terminal window. If you would prefer to run the session in the background, this can be achieved by using the –f and –N flags when initiating the connection:

$ ssh -l <username> -f -N -L 5900:localhost:5900 <remotehost>

The above command will prompt for a password for the remote server and then establish the connection in the background, leaving the terminal window available for other tasks.

If you are connecting to the remote desktop from outside the firewall keep in mind that the IP address for the SSH connection will be the external IP address provided by your ISP or cloud hosting provider, not the LAN IP address of the remote system (since this IP address is not visible to those outside the firewall). You will also need to configure your firewall to forward port 22 (for the SSH connection) to the IP address of the system running the desktop. It is not necessary to forward port 5900. Steps to perform port forwarding differ between firewalls, so refer to the documentation for your firewall, router or wireless base station for details specific to your configuration.

1.7  Establishing a Secure Tunnel on Windows using PuTTY

A similar approach is taken to establishing a secure desktop session from a Windows system to an Ubuntu server. Assuming that you already have a VNC client such as TightVNC installed, the one remaining requirement is a Windows SSH client (in this case PuTTY).

Once PuTTY is downloaded and installed, the first step is to establish a secure connection between the Windows system and the remote system with appropriate tunneling configured. When launched, PuTTY displays the following screen:

Figure 19-11

Enter the IP address or host name of the remote host (or the external IP address of the gateway if you are connecting from outside the firewall). The next step is to set up the tunnel. Click on the + next to SSH in the Category tree on the left-hand side of the dialog and click on Tunnels. The screen should subsequently appear as follows:

Figure 19-12

Enter 5900 as the Source port and localhost:5900 as the Destination and click on the Add button. Finally return to the main screen by clicking on the Session category. Enter a name for the session in the Saved Sessions text field and press Save. Click on Open to establish the connection. A terminal window will appear with the login prompt from the remote system. Enter the appropriate user login and password credentials.

The SSH connection is now established. Launch the VNC viewer, enter localhost:5900 in the VNC Server text field and click on Connect. The viewer will establish the connection, prompt for the password and then display the desktop. You are now accessing the remote desktop of a Linux system from Windows over a secure SSH tunnel connection.

1.8  Summary

Remote access to the GNOME desktop environment of an Ubuntu system can be enabled by making use of Virtual Network Computing (VNC). Comprising the VNC server running on the remote server and a corresponding client on the local host, VNC allows remote access to multiple desktop instances running on the server.

The standard remote server solution for the GNOME desktop is Vino. Once installed, remote desktop sessions can be established from other Linux systems using a remote desktop viewer such as Remmina.

When connecting from non-Linux systems such as Windows or macOS, it is necessary to disable Vino’s encryption requirements. Once disabled, connections from client systems should be established using SSH tunneling.

When the VNC connection is being used over a public connection with Vino encryption disabled, the use of SSH tunneling is recommended when connecting to ensure that the communication between client and server is encrypted and secure.

Configuring SSH Key-based Authentication on Ubuntu 20.04

When an Ubuntu system is first installed, it is not configured by default to allow remote commandline access via Secure Shell (SSH) connections. When installed, SSH provides password protected and encrypted access to the system for the root account and any other users added during the installation phase. This level of security is far from adequate and should be upgraded to SSH keybased authentication as soon as possible.

This chapter will outline the steps to increase the security of an Ubuntu system by implementing key-based SSH authentication.

1.1  An Overview of Secure Shell (SSH)

SSH is designed to allow secure remote access to systems for the purposes of gaining shell access and transferring files and data. As will be covered in “Ubuntu Remote Desktop Access with Vino”, SSH can also be used to provide a secure tunnel through which remote access to the GNOME desktop can be achieved over a network connection.

A basic SSH configuration consists of a client (used on the computer establishing the connection) and a server (running on the system to which the connection is to be established). A user might, for example, use an SSH client running on a Linux, Windows or macOS system to connect to the SSH server running on an Ubuntu system to gain access to a shell command-line prompt or to perform file transfers. All of the communications between client and server, including the password entered to gain access, are encrypted to prevent outside parties from intercepting the data.

The inherent weakness in a basic SSH implementation is that it depends entirely on the strength of the passwords assigned to the accounts on the system. If a malicious party is able to identify the password for an account (either through guess work, subterfuge or a brute force attack) the system becomes vulnerable. This weakness can be addressed by implementing SSH key-based authentication.

1.2  SSH Key-based Authentication

SSH key-based authentication makes use of asymmetric public key encryption to add an extra layer of security to remote system access. The concept of public key encryption was devised in 1975 by Whitfield Diffie and Martin Hellman and is based on the concept of using a pair of keys, one private and one public.

In a public key encryption system, the public key is used to encrypt data that can only be decrypted by the owner of the private key.

In the case of SSH key-based authentication, the private key is held by the host on which the SSH client is located while the corresponding public key resides on the system on which the SSH server is running. It is important to protect the private key, since ownership of the key will allow anyone to log into the remote system. As an added layer of protection, therefore, the private key may also be encrypted and protected by a password which must be entered each time a connection is established to the server.

1.3  Setting Up Key-based Authentication

There are four steps to setting up key-based SSH authentication which can be summarized as follows:

  1. Generate the public and private keys.
  2. Install the public key on the server.
  3. Test authentication.
  4. Disable password-based authentication on the server.

The remainder of this chapter will outline these steps in greater detail for Linux, macOS and Windows-based client operating systems.

1.4  Installing and Starting the SSH Service

If the SSH server is not already installed and running on the system, it can be added using the following commands:

# apt install openssh-server
# systemctl start sshd.service
# systemctl enable sshd.service

1.5  SSH Key-based Authentication from Linux and macOS Clients

The first step in setting up SSH key-based authentication is to generate the key pairs on the client system. If the client system is running Linux or macOS, this is achieved using the ssh-keygen utility:

# ssh-keygen

This will result in output similar to the following:

Generating public/private rsa key pair.
Enter file in which to save the key (/home/<username>/.ssh/id_rsa):

Press the Enter key to accept the default location for the key files. This will place two files in the .ssh sub-directory of the current user’s home directory. The private key will be stored in a file named id_rsa while the public key will reside in the file named id_rsa.pub.

Next, ssh-keygen will prompt for a passphrase with which to protect the private key. If a passphrase is provided, the private key will be encrypted on the local disk and the passphrase required in order to gain access to the remote system. For better security, use of a passphrase is recommended.

Enter passphrase (empty for no passphrase):

Finally, the ssh-keygen tool will generate the following output indicating that the keys have been generated:

Your identification has been saved in /home/neil/.ssh/id_rsa.
Your public key has been saved in /home/neil/.ssh/id_rsa.pub.
The key fingerprint is:
SHA256:FOLGWEEGFIjWnCT5wtTOv5VK4hdimzWghZizUEMYbfo <username>@<hostname>
The key's randomart image is:
+---[RSA 2048]----+
|.BB+=+*..        |
|o+B= * . .       |
|===.. + .        |
|*+ * . .         |
|.++ o   S        |
|..E+ * o         |
|  o B *          |
|   + +           |
|    .            |
+----[SHA256]-----+

The next step is to install the public key onto the remote server system. This can be achieved using the ssh-copy-id utility as follows:

$ ssh-copy-id username@remote_hostname 

For example:

For example:
$ ssh-copy-id neil@192.168.1.100
/usr/bin/ssh-copy-id: INFO: Source of key(s) to be installed: "/home/neil/.ssh/id_rsa.pub"
/usr/bin/ssh-copy-id: INFO: attempting to log in with the new key(s), to filter out any that are already installed
/usr/bin/ssh-copy-id: INFO: 1 key(s) remain to be installed -- if you are prompted now it is to install the new keys
neil@192.168.1.100's password: 
 
Number of key(s) added: 1
 
Now try logging into the machine, with:   "ssh 'neil@192.168.1.100'"
and check to make sure that only the key(s) you wanted were added.

Once the key is installed, test that the authentication works by attempting a remote login using the ssh client:

$ ssh -l <username> <hostname>

If the private key is encrypted and protected with a passphrase, enter the phrase when prompted to complete the authentication and establish remote access to the Ubuntu system:

Enter passphrase for key '/home/neil/.ssh/id_rsa': 
Last login: Thu Feb 21 13:41:23 2019 from 192.168.1.101
[neil@demosystem02 ~]$

Repeat these steps for any other accounts on the server for which remote access is required. If access is also required from other client systems, simply copy the id_rsa private key file to the .ssh sub-directory of your home folder on the other systems.

As currently configured, access to the remote system can still be achieved using the less secure password authentication. Once you have verified that key-based authentication works, log into the remote system, edit the /etc/ssh/ssh_config file and change the PasswordAuthentication setting to no:

PasswordAuthentication no

Save the file and restart the sshd service to implement the change:

# systemctl restart sshd.service

From this point on, it will only be possible to remotely access the system using SSH key-based authentication.

1.6  Managing Multiple Keys

It is not uncommon for multiple private keys to reside on a client system, each providing access to a different server. There are a number of options for selecting a specific key when establishing a connection. It is possible, for example, to specify the private key file to be used when launching the ssh client as follows:

$ ssh -l neilsmyth -i ~/.ssh/id_work 35.194.18.119

Alternatively, the SSH client user configuration file may be used to associate key files with servers. The configuration file is named config, must reside in the .ssh directory of the user’s home directory and can be used to configure a wide range of options including the private key file, the default port to use when connecting, the default user name, and an abbreviated nickname via which to reference the server. The following example config file defines different key files for two servers and allows them to be referenced by the nicknames home and work. In the case of the work system, the file also specifies the user name to be used when authenticating:

Host work
  HostName 35.194.18.119
  IdentityFile ~/.ssh/id_work
  User neilsmyth
 
Host home
  HostName 192.168.0.21
  IdentityFile ~/.ssh/id_home

Prior to setting up the configuration file, the user would have used the following command to connect to the work system:

$ ssh -l neilsmyth -i ~/.ssh/id_work 35.194.18.119

Now, however, the command can be shortened as follows:

$ ssh work

A full listing of configuration file options can be found by running the following command:

$ man ssh_config

1.7  SSH Key-based Authentication from Windows 10 Clients

Recent releases of Windows 10 include a subset of the OpenSSH implementation that is used by most Linux and macOS systems as part of Windows PowerShell. This allows SSH key-based authentication to be set up from a Windows 10 client using similar steps to those outlined above for Linux and macOS.

To open Windows PowerShell on a Windows 10 system press the Win+X keyboard combination and select it from the menu, or locate and select it from the Start menu. Once running, the PowerShell window will appear as shown in Figure 18-1:

Figure 18-1

If you already have a private key from another client system, simply copy the id_rsa file to a folder named .ssh on the Windows 10 system. Once the file is in place, test the authentication within the PowerShell window as follows:

$ ssh -l <username>@<hostname> 

For example:

PS C:\Users\neil> ssh -l neil 192.168.1.101
Enter passphrase for key 'C:\Users\neil/.ssh/id_rsa':

Enter the passphrase when prompted and complete the authentication process.

If an existing private key does not yet exist, generate a new private and public key pair within the PowerShell window using the ssh-keygen utility using the same steps as those outlined for Linux and macOS. Once the keys have been generated, they will once again be located in the .ssh directory of the current user’s home folder, and the public key file id_rsa.pub will need to be installed on the remote Ubuntu system. Unfortunately, Windows PowerShell does not include the ssh-copy-id utility, so this task will need to be performed manually.

Within the PowerShell window, change directory into the .ssh sub-directory and display the content of the public key id_rsa.pub file:

PS C:\Users\neil> cd .ssh
PS C:\Users\neil\.ssh> type id_rsa.pub
ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDFgx1vzu59lll6/uQw7FbmKVsQ3fzLz9MW1fgo4sdsxXp81wCHNAlqcjx1Pgr9BJPXWUMInQOi7BQ5I+vc2xQ2AS0kMq3ZH9ybWuQe/U2GjueXZd0FKrEXrT55wM36Rm6Ii3roUCoGCzGR8mn95JvRB3VtCyDdzTWSi8JBpK5gV5oOxNTNPsewlLzouBlCT1qW3CKwEiIwu8S9MTL7m3nrcaNeLewTTHevvHw4QDwzFQ+B0PDg96fzsYoTXVhzyHSWyo6H0gqrft7aKgILBtEIhWTkSVEMAzy1piKtCr1IYTmVK6engv0aoGtMUq6FnOeGp5FjvKkF4aQkh1QR28r neil@DESKTOP-S8P8D3N

Highlight the content of the file and copy it using the Ctrl-C keyboard shortcut.

Remaining within the PowerShell window, log into the remote system using password authentication:

PS C:\Users\neil\.ssh> ssh -l <username> <hostname>

Once signed in, check if the .ssh sub-directory exists. If it does not, create it as follows:

$ mkdir .ssh

Change directory into .ssh and check whether a file named authorized_keys already exists. If it does not, create it and paste the content of the public key file from the Windows 10 system into it.

If the authorized_keys file already exists it most likely already contains other keys. If this is the case, edit the file and paste the new public key at the end of the file. The following file, for example, contains two keys:

ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCzRWH27Xs8ZA5rIbZXKgxFY5XXauMv+6F5PljBLJ6j+9nkmykVe3GjZTp3oD+KMRbT2kTEPbDpFD67DNL0eiX2ZuEEiYsxZfGCRCPBGYmQttFRHEAFnlS1Jx/G4W5UNKvhAXWyMwDEKiWvqTVy6syB2Ritoak+D/Sc8nJflQ6dtw0jBs+S7Aim8TPfgpi4p5XJGruXNRScamk68NgnPfTL3vT726EuABCk6C934KARd+/AXa8/5rNOh4ETPstjBRfFJ0tpmsWWhhNEnwJRqS2LD0ug7E3yFI2qsNKGEzvAYUC8Up45MRP7liR3aMlCBil1tsy9R+IB7oMEycZAe/qj neil@localhost.localdomain
ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDFgx1vzu59lll6/uQw7FbmKVsQ3fzLz9MW1fgo4sdsxXp81wCHNAlqcjx1Pgr9BJPXWUMInQOi7BQ5I+vc2xQ2AS0kMq3ZH9ybWuQe/U2GjueXZd0FKrEXrT55wM36Rm6Ii3roUCoGCzGR8mn95JvRB3VtCyDdzTWSi8JBpK5gV5oOxNTNPsewlLzouBlCT1qW3CKwEiIwu8S9MTL7m3nrcaNeLewTTHevvHw4QDwzFQ+B0PDg96fzsYoTXVhzyHSWyo6H0gqrft7aK+gILBtEIhWTkSVEMAzy1piKtCr1IYTmVK6engv0aoGtMUq6FnOeGp5FjvKkF4aQkh1QR28r neil@DESKTOP-S8P8D3N

Once the public key is installed on the server, test the authentication by logging in to the server from within the Windows 10 PowerShell window, for example:

PS C:\Users\neil\.ssh> ssh -l neil 192.168.1.100
Enter passphrase for key 'C:\Users\neil/.ssh/id_rsa':

When key-based authentication has been set up for all the accounts and verified, disable password authentication on the Ubuntu system as outlined at the end of the previous section.

1.8  SSH Key-based Authentication using PuTTY

For Windows systems that do not have OpenSSH available, or as a more flexible alternative to using PowerShell, the PuTTY tool is a widely used alternative. The first step in using PuTTY is to download and install it on any Windows systems that need an SSH client. PuTTY is a free utility and can be downloaded using the following link:

https://www.chiark.greenend.org.uk/~sgtatham/putty/latest.html

Download the Windows installer executable that matches your Windows system (32-bit and 64bit versions are available) then execute the installer to complete installation.

If a private key already exists on another system, create the .ssh folder in the home folder of the current user and copy the private id_rsa key into it.

Next, the private key file needs to be converted to a PuTTY private key format file using the PuTTYgen tool. Locate this utility in the Windows Start menu and launch it:

Figure 18-2

Once launched, click on the Load button located in the Actions section and navigate to the private key file previously copied to the .ssh folder (note that it may be necessary to change the file type filter to All Files (*.*) in order for the key file to be visible). Once located, select the file and load it into PuttyGen. When prompted, enter the passphrase originally used to encrypt the file. Once the private key has been imported, save it as a PuTTY key file by clicking on the Save Private Key button. For consistency, save the key file to the .ssh folder but give it a different name to differentiate it from the original key file.

Launch PuTTY from the Start menu and enter the IP address or host name of the remote server into the main screen before selecting the Connection -> SSH -> Auth category in the left-hand panel as highlighted in Figure 18-3:

Figure 18-3

Click on the Browse button next to the Private key for authentication field and navigate to and select the previously saved PuTTY private key file. Optionally, scroll to the top of the left-hand panel, select the Session entry and enter a name for the session in the Saved Sessions field before clicking on the Save button. This will save the session configuration so that it can be used in future without having to re-enter the settings each time.

Finally, click on the Open button to establish the connection to the remote server, entering the user name and passphrase when prompted to do so to complete the authentication.

1.9  Generating a Private Key with PuTTYgen

The previous section explored the use of existing private and public keys when working with PuTTY. If keys do not already exist, they can be created using the PuTTYgen tool which is included with the main PuTTY installation.

To create new keys, launch PuttyGen and click on the Generate button highlighted in Figure 18-4:

Figure 18-4

Move the mouse pointer around to generate random data as instructed, then enter an optional passphrase with which to encrypt the private key. Once the keys have been generated, save the files to suitable locations using the Save public key and Save private key buttons. The private key can be used with PuTTY as outlined in the previous section. To install the public key on the remote server use the steps covered in the earlier section on using SSH within PowerShell on Windows 10.

1.10  Installing the Public Key for a Google Cloud Instance

If your Ubuntu system is hosted by Google Cloud, for example as a Compute Engine instance, there are a number of different ways to gain SSH access to the server using key-based authentication. Perhaps the most straightforward is to add your public key to the metadata for your Google Cloud account. This will make the public key available for all Virtual Machine instances that you create within Google Cloud. To add the public key, log into the Google Cloud Platform console, select the Metadata option from the left-hand navigation panel as highlighted in Figure 18-5 followed by the SSH keys tab:

Figure 18-5

On the SSH Keys screen, click on the Edit button (also highlighted in Figure 18-5) to edit the list of keys. Scroll down to the bottom of the current list and click on the + Add Item button. A new field will appear into which you will need to paste the entire public key as it appears in your id_rsa.pub file. Once the key has been entered, click on the Save button to add the key.

The public key will now appear in the list of SSH Keys. Note that the key entry also includes the username which must be used when logging into any Google Cloud instances:

Figure 18-6

With the public key added to the metadata it should be possible to access any virtual machine instance from any client on which the corresponding private key has been installed and on which the user has an account. In fact, behind the scenes, all Google Cloud has done to enable this is add the public key to the .ssh/authorized_keys file in the user’s home directory on any virtual machines on which the account exists.

1.11  Summary

It is important that any remote access to an Ubuntu system be implemented in a way that provides a high level of security. By default, SSH allows remote system access using password-based authentication. This leaves the system vulnerable to anyone who can either guess a password, or find out the password through other means. For this reason, the use of key-based authentication is recommended to protect system access. Key-based authentication uses the concept of public key encryption involving public and private keys. When implemented, users are only able to connect to a server if they are using a client which has a private key that matches a public key on the server. As an added layer of security, the private key may also be encrypted and password protected. Once key-based encryption has been implemented, the server system is then configured to disable support for the less secure password-based authentication.

This chapter has provided an overview of SSH key-based authentication and outlined the steps involved in generating keys and configuring clients on macOS, Linux and Windows, in addition to the installation and management of public keys on an Ubuntu server.