Main    /    Blog    /    Virtual Private Cloud: Creating Custom Images

←   →

Virtual Private Cloud: Creating Custom Images

10 November 2016 14 min read
Tags: , , , , , , , ,

pr-2831

The Virtual Private Cloud lets you create virtual machines from a wide-range of existing OS images.

However, some users need images which our service doesn’t offer. These include less common kinds and versions of operating systems.

Sometimes an image’s pre-installed packages or file system configuration has to be changed, like if a user is deploying a cluster of identical servers.

Instead of reconfiguring each and every server, images with those modifications can be made to speed up deployment. In this article, we’ll be looking at how exactly this is done.

In our example, we’ll be working with an Ubuntu 16.04 image. We’ll prepare the working environment, change the necessary parameters, compile it, and load our image in a cloud service.

We’ll also look at what has to be done to prepare an image that’s fully compatible with our Virtual Private Cloud and all its features.

For compiling images, we’ll be using diskimage-builder. This is a set of open-source tools, which is supported by the OpenStack community, for preparing images of operating systems, file systems, and RAM disks.

Diskimage-builder lets you create images for the most popular GNU/Linux distributions:

  • Centos
  • Debian
  • Fedora
  • RHEL
  • Ubuntu
  • Gentoo
  • OpenSUSE

By default, diskimage-builder prepares images of cloud versions of operating systems. This is why the image will include the cloud-init and cloud-utils packages, which are essential for automatically configuring systems in the cloud.

Creating an Ubuntu 16.04 Image

To start with, we’ll create an imagine with Ubuntu 14.04.

sudo apt update
sudo apt -y install python-pip curl

Then we install diskimage-builder:

sudo pip install diskimage-builder

Next, we create our base directories:

mkdir -p ~/diskimage-builder/{images,elements}

To configure diskimage-builder, we have to set some additional parameters. Their values are saved directly in the shell’s environment variables, so these can be given in the command line:

  • ARCH=”amd64” – image architecture.
  • BASE_ELEMENTS=”ubuntu bootloader cloud-init-datasources” – here we show which elements are used by diskimage-builder. Elements are sets of bash scripts that execute all of the routine actions for preparing and modifying images. For us, the most important element is ubuntu, which downloads and unpacks the standard official image of the distro, installs required packages and updates the system inside the image to the latest version using chroot, and then puts everything back together in a ready-to-use image. The bootloader element installs a boot loader (GRUB2 in our case) in our image, and cloud-init-datasources transfers a list of data sources for the cloud-init utility, which is required for the initial configuration of the OS when it’s launched.
  • DIB_CLOUD_INIT_DATSOURCES=”ConfigDrive, Ec2” – data sources for the cloud-init-datasources element.
  • DIB_RELEASE=”xenial” – the version of the operating system we’ll be creating our image from.
  • IMAGE_PATH=”~/diskimage-builder/images/ubuntu-16.04” – image name and path.

Images are compiled using the following command:

disk-image-create vm $BASE_ELEMENTS -t raw -o $IMAGE_PATH

After the -t key, we indicate the desired image format. The following formats are supported: qcow2, tar, vhd, docker, raw.

Before compiling the image, you will most likely have to make some additional modifications to the system; for example:

  • Install additional packages;
  • Turn off the predictable_interface_naming mechanism, which modifies standard interfaces names like eth0, eth1, etc. to names like enp0s3, ens3, etc.;
  • Disable creating the “ubuntu” user and allow root access over SSH;
  • Regenerate SSH keys for each virtual machine made from the image;
  • Change the pre-established time zone.

To do this, we have to create an additional element in the ~/diskimage-builder/elements directory.

We create the following directories:

mkdir -p ~/diskimage-builder/elements/ubuntu-16-custom/{install.d,post-install.d,finalise.d}

In the ~/diskimage-builder/elements/ubuntu-16-custom directory, we create a README.rst file that describes the new element:

================
ubuntu-16-custom
================
 
Customize Ubuntu 16.04 (Xenial)

In the ~/diskimage-builder/elements/ubuntu-16-custom/install.d directory, we create a script called 50-install-additional-packages:

#!/bin/bash
 
packages="python2.7
python
python-minimal"
 
apt -y install $packages

Scripts in the install.d subdirectory are executed when the image is compiled during the installation of the main packages. The 50-install-additional-packages script we create installs python ver. 2.7, which is needed to run a lot of applications. Python v. 3.5 is the default in Ubuntu 16.04.

If you need any other packages added to your image, you can add them to this list.

We then have to create a 50-configure-system script in the ~/diskimage-builder/elements/ubuntu-16-custom/post-install.d to change the system configuration:

#!/bin/bash
 
# Permit password auth via SSH
echo 'ssh_pwauth: true'     	>> /etc/cloud/cloud.cfg.d/50_remote_access.cfg
# Generate new keys in first boot
echo 'ssh_deletekeys: true' 	>> /etc/cloud/cloud.cfg.d/50_remote_access.cfg
# Don’t disable root access via SSH
echo 'disable_root: false'  	>> /etc/cloud/cloud.cfg.d/50_remote_access.cfg
# Don’t create "ubuntu" user
echo 'users: []'            	>> /etc/cloud/cloud.cfg.d/50_remote_access.cfg
# Change default timezone to MSK
echo 'timezone: Europe/Moscow'  >> /etc/cloud/cloud.cfg.d/50_timezone.cfg
# Change PermitRootLogin value to "yes"
sed -i 's/PermitRootLogin .*/PermitRootLogin yes/g' /etc/ssh/sshd_config
# Set root password to empty
passwd -d root

Scripts in the post-install.d directory are executed right after the scripts in the install.d directory.

We also have to change our GRUB settings to add the “net.ifnames=0” launch parameter, which saves interface names in standard ethN format.

We add the 50-configure-grub script to the ~/diskimage-builder/elements/ubuntu-16-custom/finalise.d directory:

#!/bin/bash
 
sed -i 's/\(^GRUB_CMDLINE_LINUX.*\)"$/\1 net.ifnames=0"/' /etc/default/grub
 
update-grub

Scripts in the finalise.d directory are executed after all the main phases have run.

After making these scripts, we have to change their attributes. This is done with the chmod command:

chmod 0755 ~/diskimage-builder/elements/ubuntu-16-custom/*d/*

Before creating our image, we have to give the path of our new element. We do this in the ELEMENTS_PATH parameter.

To avoid having to define parameters directly in the command line each time, we’ll create a build-ubuntu-16.04 script in the ~/diskimage-builder/ directory, which will define the parameters for diskimage-builder and compile the image:

#!/bin/bash
 
export ARCH="amd64"
export BASE_ELEMENTS="bootloader cloud-init-datasources ubuntu ubuntu-16-custom"
export DIB_CLOUD_INIT_DATASOURCES="ConfigDrive, Ec2"
export DIB_RELEASE="xenial"
export ELEMENTS_PATH="./elements/:/usr/share/diskimage-builder/elements/"
export IMAGE_PATH="./images/ubuntu-16.04"
 
disk-image-create vm $BASE_ELEMENTS -t raw -o $IMAGE_PATH

We change the attributes of this file:

chmod 0755 ~/diskimage-builder/build-ubuntu-16.04

Afterwards, we should get this file architecture:

$tree -p ~/diskimage-builder
/home/user/diskimage-builder
├── [-rwxr-xr-x]  build-ubuntu-16.04
├── [drwxr-xr-x]  elements
│   └── [drwxr-xr-x]  ubuntu-16-custom
│   	├── [drwxr-xr-x]  finalise.d
│   	│   └── [-rwxr-xr-x]  50-configure-grub
│   	├── [drwxr-xr-x]  install.d
│   	│   └── [-rwxr-xr-x]  50-install-additional-packages
│   	├── [drwxr-xr-x]  post-install.d
│   	│   └── [-rwxr-xr-x]  50-configure-cloud-init
│   	└── [-rw-r--r--]  README.rst
└── [drwxr-xr-x]  images

We run the build-ubuntu-16.04 script:

cd ~/diskimage-builder
sudo ./build-ubuntu-16.04

We need to use sudo to keep the application from requesting a password while preparing the image. This is because some of diskimage-builder’s pre-installed elements contain the sudo call (like the 01-ccache script in the base element, which is executed while compiling most distributions).

After our script has run, a new image will be found in the ~/diskimage-builder/images/ubuntu-16.04.raw directory.

You can use this image with Selectel’s Virtual Private Cloud by uploading it in the VPC web interface or from the glance API.

Keep in mind that the root password was deleted when the image was being compiled, so SSH connections to virtual machines can initially only be made with a key. SSH keys can be added in the VPC panel when you create your machine. Servers can also be accessed from the no-VNC virtual console in the control panel. If no root password has been set, you will not have to enter one.

The root password can be set normally with the passwd utility.

If you use an image in our VPC that was prepared in the way we’ve described above, you’ll be met with several limitations. The following VPC features will not be available:

  • setting service-generated system passwords;
  • changing the root password from the VPC web panel;
  • changing the root password from the nova API;
  • changing the default route interface from the VPC web panel;
  • automatically changing network interface settings inside the system if they were changed by the server in the control panel.

In the next section, we’ll show you how to create a fully-compatible VPC image.

Creating Images for the Virtual Private Cloud

For maximum compatibility with the VPC, several properties must be added to the image and packages to the system while it’s being compiled.

We’ll add the ubuntu-16-selectel element by creating the directory:

mkdir -p ~/diskimage-builder/elements/ubuntu-16-selectel/

Inside this directory, we add a description of the element in the README.rst file:

====================
ubuntu-16-selectel
====================
 
Build ubuntu image for Virtual Private Cloud

We’ll create our directories for the new element:

mkdir -p ~/diskimage-builder/elements/ubuntu-16-selectel/{pre-install.d,install.d,post-install.d}

Next, we change the repository list with the script ~/diskimage-builder/elements/ubuntu-16-selectel/pre-install.d/50-add-mirrors:

#!/bin/bash
 
# Add key for Selectel OpenStack repository
apt-key adv --fetch-keys http://repo.os.selectel.org/selectel-openstack.key
 
cat < /etc/apt/sources.list
## Selectel OpenStack repository
deb http://repo.os.selectel.org xenial main
 
## Selectel mirrors
deb http://mirror.selectel.ru/ubuntu xenial main restricted universe multiverse
deb http://mirror.selectel.ru/ubuntu xenial-updates main restricted universe multiverse
deb http://mirror.selectel.ru/ubuntu xenial-backports main restricted universe multiverse
 
## Security updates
deb http://security.ubuntu.com/ubuntu xenial-security main restricted universe multiverse
EOF

We’ll add additional packages with the script ~/diskimage-builder/elements/ubuntu-16-selectel/install.d/50-add-selectel-packages:

#!/bin/bash
 
packages="crontab-randomizer
fstrim-blocks
qemu-guest-agent
set-root-pw"
 
apt -y install $packages

This will install the following utilities:

  • crontab-randomizer – a simple script that randomizes time intervals in /etc/crontab, we’ll have to add this script’s call when the system is first launched;
  • fstrim-blocks – a wrapper for the pre-installed fstrim utility, which performs a block-sized TRIM of the file system on fast disks in a short interval, adds a call to cron.weekly during installation;
  • qemu-guest-agent – a utility which lets you sends QMP commands to virtual machines to manage the system, in our case, this utility is needed to change passwords on virtual machines from the VPC panel and nova API;
  • set-root-pw – a script to initially set a root password on virtual machines from OpenStack metadata, we’ll add this script’s call when the script is first launched.

When the image is being compiled, the cloud-init package will be automatically updated to the version in the Selectel OpenStack mirror. This version contains patches that ensure network consistency between virtual machines and the VPC service.

Packages are automatically updated during the install.d stage using the pre-installed 00-up-to-date script.

Next, we have to add additional configuration files for cloud-init. In the ~/diskimage-builder/elements/ubuntu-16-selectel/post-install.d, we’ll create a 50-configure-cloud-init script:

#!/bin/bash
 
# Prevent cloud-init to change custom apt mirrors
echo 'apt_preserve_sources_list: true'  	> /etc/cloud/cloud.cfg.d/50_selectel_mirror.cfg             	
echo 'system_info:'                    	>> /etc/cloud/cloud.cfg.d/50_selectel_mirror.cfg                                                    	
echo '- arches: [i386, amd64]'         	>> /etc/cloud/cloud.cfg.d/50_selectel_mirror.cfg                              	
echo 'search:'                         	>> /etc/cloud/cloud.cfg.d/50_selectel_mirror.cfg                                                         	
echo 'primary:'                        	>> /etc/cloud/cloud.cfg.d/50_selectel_mirror.cfg                                                     	
echo '- http://mirror.selectel.ru/ubuntu'  >> /etc/cloud/cloud.cfg.d/50_selectel_mirror.cfg    	
echo 'failsafe:'                       	>> /etc/cloud/cloud.cfg.d/50_selectel_mirror.cfg
echo '- http://archive.ubuntu.com/ubuntu/' >> /etc/cloud/cloud.cfg.d/50_selectel_mirror.cfg
 
# Prevent cloud-init to disable EC2-style metadata
echo 'disable_ec2_metadata: false' > /etc/cloud/cloud.cfg.d/50_enable_ec2.cfg
 
# Perform some commands in boot time
# 'runcmd' only runs during the first boot
echo 'runcmd:'                 	> /etc/cloud/cloud.cfg.d/50_first_boot_routines.cfg                             	
echo '- set-root-pw 2> /dev/null' >> /etc/cloud/cloud.cfg.d/50_first_boot_routines.cfg
echo '- crontab-randomizer'   	>> /etc/cloud/cloud.cfg.d/50_first_boot_routines.cfg

The pre-installed script /etc/cron.weekly/fstrim should also be deleted if fstrim-blocks is used, since the pre-installed option does not contain parameters for block launching, which puts additional workload on the system when fstrim is launched.

We delete the file with the simple script ~/diskimage-builder/elements/ubuntu-16-selectel/post-install.d/51-remove-fstrim-weekly:

#!/bin/bash
 
rm -f /etc/cron.weekly/fstrim

We change the attributes of the new scripts:

chmod 755 ~/diskimage-builder/elements/ubuntu-16-selectel/*d/*

Next, we add a new element, ubuntu-16-selectel, to the script ~/diskimage-builder/build-ubuntu-16.04:

#!/bin/bash
 
export ARCH="amd64"
export BASE_ELEMENTS="bootloader cloud-init-datasources ubuntu ubuntu-16-custom ubuntu-16-selectel"
export DIB_CLOUD_INIT_DATASOURCES="ConfigDrive, Ec2"
export DIB_RELEASE="xenial"
export ELEMENTS_PATH="./elements/:/usr/share/diskimage-builder/elements/"
export IMAGE_PATH="./images/ubuntu-16.04"
 
disk-image-create vm $BASE_ELEMENTS -t raw -o $IMAGE_PATH

The file hierarchy should look like this:

$ tree -p diskimage-builder
diskimage-builder
├── [-rw-r--r--]  build-ubuntu-16.04
└── [drwxr-xr-x]  elements
    ├── [drwxr-xr-x]  ubuntu-16-custom
    │   ├── [drwxr-xr-x]  finalise.d
    │   │   └── [-rwxr-xr-x]  50-configure-grub
    │   ├── [drwxr-xr-x]  install.d
    │   │   └── [-rwxr-xr-x]  50-install-additional-packages
    │   ├── [drwxr-xr-x]  post-install.d
    │   │   └── [-rwxr-xr-x]  50-configure-system
    │   └── [-rw-r--r--]  README.rst
    └── [drwxr-xr-x]  ubuntu-16-selectel
        ├── [drwxr-xr-x]  install.d
        │   └── [-rwxr-xr-x]  50-add-selectel-packages
        ├── [drwxr-xr-x]  post-install.d
        │   └── [-rwxr-xr-x]  50-configure-cloud-init
        ├── [drwxr-xr-x]  pre-install.d
        │   └── [-rwxr-xr-x]  50-add-selectel-openstack-mirror
        └── [-rw-r--r--]  README.rst

Now we compile the image:

cd ~/diskimage-builder
sudo ./build-ubuntu-16.04

To load the image and all of its properties, you will need glance and the RC file to access the project (this can be found along with all other instructions in our control panel at https://support.selectel.ru/vpc/access).

The load command should look something like this:

glance image-create --name Ubuntu-16.04-VPC \
--disk-format raw \
--container-format bare \
--property hw_disk_bus=scsi \
--property hw_scsi_model=virtio-scsi \
--property x_sel_image_owner=Selectel \
--property hw_qemu_guest_agent=yes \
--file ~/diskimage-builder/images/ubuntu-16.04.raw \
--progress

While it loads, the following properties will be added:

  • hw_qemu_guest_agent=yes – required to support the mechanism for changing server passwords from the VPC panel or nova API; this property will only work if the qemu_guest_agent utility in the image is version 2.3 or above (Ubuntu 16.04 uses version 2.5);
  • x_sel_image_owner=Selectel – required for viewing passwords in the VPC panel;
  • hw_disk_bus=scsi, hw_scsi_model=virtio-scsi – this lets you use more modern and high-performance virtio-scsi buses, which give you the ability to use TRIM.

Once the image has been loaded, it will appear under the Images tab in your VPC project with the name Ubuntu-16.04-VPC.

During the server’s initial launch, a root password will be generated and automatically set. It will appear under the Console tab. Here, new passwords can be generated if need be.

Additionally, network interfaces won’t have to be manually configured on your server if you change them in the control panel under Ports. For cloud-init to configure network interfaces, a hard-restart will have to be performed on that server. Once it boots up, new OpenStack metadata will be generated with new network configurations.

You can keep cloud-init from changing network configurations each time the server restarts by adding the file /etc/cloud/cloud.cfg.d/99_disable_network_config.cfg to the system:

network:
    config: disabled

In the near future, we’ll add the ability to disable automatic network reconfigurations from the VPC control panel and without having to modify the system configuration.

Conclusion

In this article, we looked at diskimage-builder’s basic abilities, how to add our own scripts to change image configurations, the main stages of compiling images, and an example of how to load an image in a cloud environment.

If you have any questions regarding the main ways to use this tool or if anything about managing images and virtual machines in the VPC is unclear, please let us know in the comments below.

We’d love to hear how you use diskimage-builder.

10 November 2016 Tags: , , , , , , , ,

Popular articles